Reference picture set signaling and restriction on an electronic device

ABSTRACT

A previously signaled reference picture set (RPS) corresponding to a current picture is indicated. A first flag for a picture in the previously signaled RPS is set if the picture is to be used as a reference picture for the current picture. A bitstream is sent.

CROSS REFERENCE TO RELATED APPLICATIONS

This application is a continuation-in-part of U.S. patent applicationSer. No. 13/355,472, entitled “DECODING A PICTURE BASED ON A REFERENCEPICTURE SET ON AN ELECTRONIC DEVICE” filed on Jan. 20, 2012, which is acontinuation-in-part of U.S. patent application Ser. No. 13/354,277,entitled “REDUCING REFERENCE PICTURE SET SIGNAL OVERHEAD ON ANELECTRONIC DEVICE,” filed on Jan. 19, 2012, both which are herebyincorporated by reference herein, in their entirety; and claims thebenefit of Provisional Application No. 61/809,201, entitled “REFERENCEPICTURE SET SIGNALING AND RESTRICTION ON AN ELECTRONIC DEVICE,” filedApr. 5, 2012.

TECHNICAL FIELD

The present disclosure relates generally to electronic devices. Morespecifically, the present disclosure relates to inter reference pictureset prediction and signaling on an electronic device.

BACKGROUND OF THE INVENTION

Electronic devices have become smaller and more powerful in order tomeet consumer needs and to improve portability and convenience.Consumers have become dependent upon electronic devices and have come toexpect increased functionality. Some examples of electronic devicesinclude desktop computers, laptop computers, cellular phones, smartphones, media players, integrated circuits, etc.

Some electronic devices are used for processing and displaying digitalmedia. For example, portable electronic devices now allow for digitalmedia to be consumed at almost any location where a consumer may be.Furthermore, some electronic devices may provide download or streamingof digital media content for the use and enjoyment of a consumer.

The increasing popularity of digital media has presented severalproblems. For example, efficiently representing high-quality digitalmedia for storage, transmittal and playback presents several challenges.As can be observed from this discussion, systems and methods thatrepresent digital media more efficiently may be beneficial.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram illustrating an example of one or moreelectronic devices in which systems and methods for reducing referencepicture set (RPS) signal overhead may be implemented;

FIG. 2 is a block diagram illustrating one configuration of a group ofpictures (GOP);

FIG. 3 is a block diagram illustrating one configuration of an encoderon an electronic device;

FIG. 4 is a flow diagram illustrating one configuration of a method forreducing RPS signal overhead on an electronic device;

FIG. 5 is a flow diagram illustrating a more specific configuration of amethod for reducing RPS signal overhead on an electronic device;

FIG. 6 is a flow diagram illustrating another more specificconfiguration of a method for reducing RPS signal overhead on anelectronic device;

FIG. 7 is a block diagram illustrating one configuration of a decoder onan electronic device;

FIG. 8 is a flow diagram illustrating one configuration of method forreducing RPS signal overhead on an electronic device;

FIG. 9 is a flow diagram illustrating one configuration of a method forderiving a partial RPS on an electronic device;

FIG. 10 is a flow diagram illustrating one configuration of a method forderiving an RPS template on an electronic device;

FIG. 11 is a flow diagram illustrating one configuration of a method forcreating an indication to delete at least one reference picture on anelectronic device;

FIG. 12 is a flow diagram illustrating one configuration of a method fordeleting at least one reference picture on an electronic device;

FIG. 13 illustrates various components that may be utilized in anelectronic device;

FIG. 14 is a flow diagram illustrating one configuration of a method fordecoding a picture on an electronic device;

FIG. 15 is a flow diagram illustrating a more specific configuration ofa method for decoding a picture on an electronic device;

FIG. 16 is a flow diagram illustrating another more specificconfiguration of a method decoding a picture on an electronic device;

FIG. 17 is a flow diagram illustrating one configuration of a method forsignaling a reference picture set on an electronic device;

FIG. 18 is a flow diagram illustrating another configuration of a methodfor signaling a reference picture set on an electronic device;

FIG. 19 is a flow diagram illustrating a more specific configuration ofa method for signaling a reference picture set on an electronic device;

FIG. 20 is a flow diagram illustrating one configuration of a method forpredicting a reference picture set on an electronic device;

FIG. 21 is a flow diagram illustrating another configuration of a methodfor predicting a reference picture set on an electronic device; and

FIG. 22 is a flow diagram illustrating a more specific configuration ofa method for predicting a reference picture set on an electronic device.

FIG. 23 is a block diagram illustrating two configurations of differentpicture types.

FIG. 24 is a block diagram illustrating a inter-layer reference pictureset creation module.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

A method for signaling a reference picture set on an electronic deviceis described. A previously signaled reference picture set (RPS)corresponding to a current picture is indicated. A first flag for apicture in the previously signaled RPS is set if the picture is to beused as a reference picture for the current picture. A first flag for apicture in the previously signaled RPS is not set if the picture is notto be used as a reference picture for the current picture. A bitstreamis sent.

A used by current picture flag for a current RPS may not be sent. Thepreviously signaled RPS may be signaled as an index into a set ofpreviously signaled RPSs.

The previously signaled RPS may include at least one corresponding valueof another reference picture in the previously signaled RPS. The atleast one corresponding value may be a used by current picture flagvalue of a second reference picture in the previously signaled RPS. Thepreviously signaled RPS may be indicated by a predicted short-termreference picture set present flag.

The first flag may be a keep picture flag. The current picture may be ina first group of pictures (GOP) after a random access point. Thepreviously signaled RPS may be signaled via a delete operation.

The previously signaled RPS may be sent from one of a picture parameterset (PPS), a sequence parameter set (SPS), an adaptation parameter set(APS), a video parameter set (VPS) and a slice header (SH). Thepreviously signaled RPS may be sent from an element in the bitstream.

An electronic device for signaling a reference picture set is alsodescribed. The electronic device includes a processor and executableinstructions stored in memory that is in electronic communication withthe processor. The electronic device indicates a previously signaledreference picture set (RPS) corresponding to a current picture. Theelectronic device also sets a first flag for a picture in the previouslysignaled RPS if the picture is to be used as a reference picture for thecurrent picture. The electronic device further does not set a first flagfor a picture in the previously signaled RPS if the picture is not to beused as a reference picture for the current picture. The electronicdevice further sends a bitstream.

A method for decoding a reference picture set on an electronic device isalso described. A bitstream is obtained. A current picture is obtained.A previously signaled reference picture set (RPS) is obtained. For apicture in the previously signaled RPS, a first flag indication isobtained if the picture should be used as a reference picture for thecurrent picture. If the first flag is positive, a second flag from atleast one corresponding value of the previously signaled RPS isdetermined. The current picture is decoded.

Decoding the current picture may be based on the second flag. The firstflag may be a keep picture flag. The second flag may be a used bycurrent picture flag.

The at least one corresponding value may be obtained from the previouslysignaled RPS. The at least one corresponding value may be a used bycurrent picture flag value of a second reference picture in thepreviously signaled RPS. The at least one corresponding value may be aBoolean value.

The current picture may be in a first group of pictures (GOP) after arandom access point. The previously signaled RPS may be signaled via adelete operation. The previously signaled RPS may be obtained from oneof a picture parameter set (PPS), a sequence parameter set (SPS), anadaptation parameter set (APS), a video parameter set (VPS) and a sliceheader (SH). The previously signaled RPS may be obtained from an elementin the bitstream.

An electronic device for decoding a reference picture set is alsodescribed. The electronic device includes a processor and executableinstructions stored in memory that is in electronic communication withthe processor. The electronic device obtains a bitstream. The electronicdevice also obtains a current picture. The electronic device furtherobtains a previously signaled reference picture set (RPS). For a picturein the previously signaled RPS, the electronic device obtains a firstflag indication if the picture should be used as a reference picture forthe current picture. If the first flag is positive, the electronicdevice determines a second flag from at least one corresponding value ofthe previously signaled RPS. The electronic device additionally decodesthe current picture.

The systems and methods disclosed herein describe several configurationsfor reducing reference picture set (RPS) signal overhead on anelectronic device. For example, the systems and methods disclosed hereindescribe encoding and decoding an RPS. For instance, several approachesfor decoding an RPS are described. Additionally, approaches for encodingan RPS to achieve reduced signal overhead are also described.

A RPS is a set of reference pictures associated with a picture. An RPSmay include reference pictures that are prior to the associated picturein decoding order that may be used for inter prediction of theassociated picture and/or for any picture following the associatedpicture in decoding order. An RPS describes one or more referencepictures in the decoded picture buffer (DPB). This is accomplished inthe slice header of each picture. Previous video coding standards, suchas H.264/AVC referenced reference pictures in a relative manner. Anypictures in the DPB that are not a part of the reference picture set maybe marked as “unused for reference.”

A DPB may be used to store reconstructed (e.g., decoded) pictures at adecoder. These stored pictures may then be used, for example, in aninter-prediction mechanism. When pictures are decoded out of order, thepictures may be stored in the DPB so they can be displayed later inorder. Also, a picture in the DPB may be associated with a picture ordercount (POC). The POC may be a variable that is associated with eachencoded picture and that has a value that increases with increasingpicture position in an output order. In other words, the POC may be usedby the decoder to deliver the pictures in the correct order for display.The POC may also be used for identification of reference pictures duringreference picture list construction and decoded reference picturemarking.

In some configurations, reference pictures are referenced using eitherrelative (e.g., delta) referencing (using a deltaPOC and a currentPOC,for example) or absolute referencing (using the POC, for example). Forinstance, the DPB may contain a set of received pictures. A subset ofthese received pictures may use relative (e.g., delta) referencing andthe remaining received pictures may use absolute referencing. It shouldbe noted that one or more of the configurations of buffer descriptionsand syntaxes described herein may be implemented in combination with oneor more of the approaches (e.g., methods) described herein.

A RPS may contain a list of information of all reference pictures thatthe decoder shall keep. For example, this information may be stored as aset of indexes called deltaPOCs. A deltaPOC may be used to calculate thePOC of a reference picture. For instance,POC_reference=POC_current+deltaPOC. In other words, by using the currentPOC of the picture to be decoded and the deltaPOC of a referencepicture, the reference picture may be located in a relative manner.Additionally an RPS may store a temporal ID for each reference pictureand/or a flag which indicates if the particular reference picture isused by the current picture.

An example of how an RPS works at on an electronic device follows.Suppose an Inter-frame (I-frame) picture is received followed twobidirectional predicted (B-frame) pictures, then followed by twobidirectional predicted (b-frame) pictures. A B-frame is a bidirectionalpredicted picture that is used for prediction by other pictures. Ab-frame is a bidirectional predicted picture that is not used forprediction by other pictures.

In other words, the order of receiving the pictures is I₀-B₁-B₂-b₁-b₂.In this example, the GOP size is 4.

Further suppose that the I-frame has a POC of 0, the first receivedb-frame has a POC of 1, the second received B-frame has a POC of 2, thefirst received b-frame has a POC of 3 and the first received B-frame hasa POC of 4. In other words, the POC order is I₀-b₁-B₂-b₂-B₁.Additionally, suppose that the I-frame serves as a reference picture forB₁, B₂ and b₁; B₁ serves as a reference picture for B₂, and b₂; and B₂serves as a reference picture for b₁ and b₂.

In this example, the second picture in decoding order (e.g., pictureB₁/POC 4) will include picture of POC 0 (e.g., I₀) in its RPS. To storeI₀/POC 0, B1 may write deltaPOC=−4 into its RPS index value. In otherwords, the difference of the reference POC relative to current POC isstored in the RPS as an index value.

The third picture in decoding order (i.e., picture B₂/POC 2) willinclude pictures of POC 0 and POC 4 (e.g., B₁) in its RPS. Thus, theindex values of deltaPOC=−2 and 2 are stored in its RPS.

Continuing the example, the fourth picture in decoding order (i.e.,picture b₂/POC 1) may include both pictures of POC 0 (e.g., I₀) and POC2 (e.g., B₂) in its RPS. Further, b₂/POC 1 may also include POC 4 (e.g.,B₁) since that picture will be used for reference in the future. Here,deltaPOCs −1, 1, 3 are stored in its RPS. It should be noted that bothpositive and negative deltaPOCs may be stored in an RPS.

Finishing this example, the 5th picture in decoding order ((i.e.,picture b₁/POC 3) may include pictures of POC 2 (e.g., B₂) and POC 4(e.g., B₁) in its RPS. Thus, deltaPOC=−1 and 1 are the relative valuesstored in b₁'s RPS as index values. It may also be noted that b₁ doesnot need to include POC 0 (e.g., I₀) in its RPS unless I₀ is going to beused for reference in the future. If POC 0 (e.g., I₀) is not included inthe RPS of the b₁, it may be marked as “unused for reference.”

Once an RPS has been listed and constructed, it is ready to be signaled.There are various ways to signal an RPS. According to one approach, aset of templates associated with handling RPSs are signaled in thepicture parameter set (PPS) and referred to by each slice with an RPSindex in a slice header. Under another method, an RPS may be signaledexplicitly in a slice header.

Listing (1) below shows one example of syntax for signaling the RPS inthe PPS.

Listing (1) /* Reference picture set syntax for when the RPS is signaledin the PPS /* Descriptors used in all listings: ue(v): Unsigned integer,entropy coded variable length u(x): Unsigned x-bit(s) integer */pic_parameter_set ( ) {   pic_parameter_set_id   seq_paramater_set_id  entropy_coding_mode_flag   num_ref_pic_sets   for(idx=0; idx <num_ref_pic_sets_idx++)     ref_pic_set(idx)   ... } ref_pic_set(idx){  num_negative_pics   num_positive_pics   for(i = 0; i <num_negative_pics; i++){     delta_poc_s0_minus1[i]    used_by_curr_pic_s0_flag[i]   }   for(i = 0; i < num_positive_pics;i++){     delta_poc_s1_minus1[i]     used_by_curr_pic_s1_flag[i]   } }

partial_ref_pic_set_flag[i] indicates to use full RPS[i] to generate apartial RPS. Ref_flag specifies which reference index of the full RPS[i]is copied into the partial RPS.

seq_parameter_set_id identifies the sequence parameter set that isreferred to by the picture parameter set. The value ofseq_parameter_set_id shall be in the range of 0 to 31, inclusive.

pic_parameter_set_id identifies the picture parameter set that isreferred to in the slice header. The value of pic_parameter_set_id shallbe in the range of 0 to 255, inclusive. entropy_coding_mode_flagindicates the entropy decoding method to be applied for the syntaxelements.

num_(—) ref_pic_sets specifies the number of reference picture sets thatare specified in the picture parameter set. num_negative_pics specifiesthe number of the following delta_poc_s0_minus1[i] andused_by_curr_pic_s0_flag[i] syntax elements. num_positive_pics specifiesthe number of the following delta_poc_s1_minus1[i] andused_by_curr_pic_s1_flag1[i] syntax elements. delta_poc_s0_minus1[i]plus 1 specifies an absolute difference between two picture order countvalues.

used_by_curr_pic_s0_flag[i] equal to 0 specifies that the i^(th)reference picture that has picture order count less than that of thecurrent picture is not used for reference by the current picture.delta_poc_s1_minus1[i] plus 1 specifies an absolute difference betweentwo picture order count values. used_by_curr_pic_s1_flag[i] equal to 0specifies that the i^(th) reference picture that has picture order countgreater than that of the current picture is not used for reference bythe current picture.

It should be noted that the definitions given for parameters used inListing (1) may be applied to all listings given herein. Furthermore,parameter definitions be given in subsequent listings may also apply topreviously listed listings.

Listing (2) shows one example of syntax for signaling the RPS in a sliceheader.

Listing (2) /*  Syntax for signaling the RPS in a slice header. */slice_header ( ) {   ...   if(IdrPicFlag){     ...   }   else {     ...    ref_pic_set_pps_flag     if(!ref_pic_set_pps_flag)      ref_pic_set(num_ref_pic_sets)     else       ref_pic_set_idx    ...   } }

ref_pic_set_pps_flag equal to 1 specifies that the reference picture setof the current picture shall be created using syntax elements in theactive picture parameter set. ref_pic_set_pps_flag equal to 0 specifiesthat the reference picture set of the current picture shall be createdusing syntax elements in the ref_pic_set( ) syntax structure in theslice header.

ref_pic_set_pps_idx specifies the index to the list of reference picturesets specified in the active picture parameter set that shall be usedfor creation of the reference picture set of the current picture.

Further, RPS templates may be signaled to a decoder to assist indecoding a picture. Templates may be for a Random Access common testcondition where eight pictures are grouped together (e.g., group ofpictures (GOP)). Table (1) below shows an RPS template for Random Accesscommon test condition (GOP=8). Some terms have been abbreviated forconvenience in Tables (1)-(4). These terms are abbreviated as follows:Temporal_id (TId), ref_buf_size (RBS) and RPS Index Values (RPS IndexValues).

TABLE (1) RPS RPS # Type POC TId RBS ref_pic #ref_pics Idx Values 1: B 80 4 1 4 −8 −10 −12 −16 2: B 4 0 2 1 3 −4 −6 4 3: B 2 0 2 1 4 −2 −4 2 64: B 1 0 2 0 4 −1 1 3 7 5: B 3 0 2 0 4 −1 −3 1 5 6: B 6 0 2 1 4 −2 −4 −62 7: B 5 0 2 0 4 −1 −5 1 3 8: B 7 0 2 0 4 −1 −3 −7 1

In Table (1), Type signifies the type of frame used. In this case, allFrames are B-frames. Note, Frame 0, which is not shown, is an I-frame.In Table (1), Temporal_id (TId) specifies the temporal layer ID of thisframe, ref_buf_size (RBS) is the reference buffer size needed by thecurrent picture, ref_pic indicates if the frame is a reference picturefor other pictures, #ref_pics is the number of reference pictures andRPS Index Values (RPS Idx Values) represent the index values of thereference pictures stored in each RPS for each corresponding frame.

An additional partial RPS may be generated for frames that have one ormore unavailable reference pictures due to instantaneous decodingrefresh (IDR) or clean random access (CRA). Table (2) shows a templatefor a partial RPS generated for a Random Access common test condition.

TABLE (2) RPS RPS # Type POC TId RBS ref_pic #ref_pics Idx Values 1: B 80 4 1 1 −8 2: B 4 0 2 1 2 −4 4 3: B 2 0 2 1 3 −2 2 6

A partial RPS may be created from a full RPS. In Table (1), the RPSindex for Frame 1 is −8. −10, −12 and −16. The RPS index for the partialRPS for Frame 1 in Table (2) is only −8. Similarly, Frames 2 and 3 inthe partial RPS have a lesser number of reference pictures in Table (2)than in the full RPS shown in Table (1). Thus, in this example, apartial RPS contains fewer RPS index values per frame than a full RPS.

Table (3) shows the template for a Low Delay common test condition,where GOP equals 4.

TABLE (3) RPS RPS # Type POC TId RBS ref_pic #ref_pics Idx Values 1 B 10 4 1 4 −1 −5 −9 −13 2 B 2 0 4 1 4 −1 −2 −6 −10 3 B 3 0 4 1 4 −1 −3 −7−11 4 B 4 0 4 1 4 −1 −4 −8 −12

In Table (3), each GOP has four pictures. In the template, there arealso four RPSs that correspond to pictures at each POC position:

RPS[0], which has index values of [−1,−5,−9,−13] for POC 1;

RPS[1], which has index values of [−1 −2 −6 −10] for POC 2;

RPS[2], which has index values of [−1, −3,−7,−11] for POC 3; and

RPS[3], which has index values of [−1,−4,−8,−12] for POC 4;

for a sequence 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, etc.

Table (4) shows a template for a partial RPS generated for a Low Delaycommon test condition. Table (4) is a continuation of Table (3).

TABLE (4) RPS RPS# Type POC TId RBS ref_pic #ref_pics Idx Values 5 B 1 04 1 1 −1 6 B 2 0 4 1 2 −1 −2 7 B 3 0 4 1 3 −1 −2 −3 8 B 4 0 4 1 4 −1 −2−3 −4 9 B 1 0 4 1 4 −1 −2 −3 −5 10 B 2 0 4 1 4 −1 −2 −3 −6 11 B 3 0 4 14 −1 −2 −3 −7 12 B 4 0 4 1 4 −1 −2 −4 −8 13 B 1 0 4 1 4 −1 −2 −5 −9

In Table (4), additional index references are generated for frames thathave unavailable reference pictures. A reference picture may beunavailable due to IDR or CRA. When generating a partial RPS for the LowDelay common test case, pictures for the first few frames in the GOPafter an IDR or CRA are selected. For example, Frames 1 to 4 at start ofa sequence belong to the first GOP, it will use partial RPS[5] toRPS[8], which are derived from RPS[0] to RPS[3]. Frames 5 to 8 belong tothe second GOP and will use partial RPS[9] to RPS[12,] which are alsoderived from RPS[0] to RPS[3]. Frames 9 belongs to the third GOP andwill use partial RPS[13] which is derived from RPS[0].

In some configurations, the electronic device deriving the partial RPSmay employ rules to derive partial the RPS from the full RPS. Forexample, RPS Numbers 7, 8, 9, 11, 12 and 13 may all add the RPS indexvalue −2 (indicated in bold). Note that this RPS index value is relativeand is dependent on the current frame position (e.g., current POC).Further, RPS Numbers 8, 9 and 10 each add the RPS index value of −3(indicated in bold).

According to known approaches, such as that specified in High EfficiencyVideo Coding (HEVC) test model (HM) 5.0, both the full RPS and partialRPS are signaled in the bitstream. For example, RPS templates, includinga full RPS and a partial RPS are sent in a PPS at the start of picture.This leads to unnecessary overhead being sent in the bitstream. Thisoverhead of signaling RPS templates in PPS may be from several hundredto several thousands of bits, for example. Further, extra andunnecessary work is also performed at the encoder to generate a partialRPS. Thus, one of the benefits on the systems and methods disclosedherein is that RPS signaling overhead is reduced by generating partialRPS templates at the decoder side based on a received full RPS.

Also, according to known approaches, such as that specified in HM5.0,within one RPS, reference picture indexes are differentially coded amongthe negative indexes and among the positive indexes. Thus, it may bebeneficial to reduce the number of signaled bits by employing thesystems and methods disclosed herein to reduce the number bits for anRPS template. For example, this may be accomplished by reducing thenumber of bits required to code a list of RPS index values.

The systems and methods disclosed herein may provide one or moreadditional benefits in reducing RPS signal overhead. In oneconfiguration, symmetry characteristics of positive and negativereference indexes may be employed to reduce the bits used for codingpositive RPS index values (e.g., reference indexes). Under anotherconfiguration, a partial RPS may be derived at the decoder rather thansending the partial RPS in the bitstream. Additionally or alternatively,frequently used RPS templates at both the encoder and decoder side maybe defined, allowing the template to be signaled by a simple index inthe PPS. Additionally or alternatively, a partial RPS may be signaled byreferring to a full RPS and deleting some of the reference pictures toarrive at the partial RPS.

Various configurations are now described with reference to the Figures,where like reference numbers may indicate functionally similar elements.The systems and methods as generally described and illustrated in theFigures herein could be arranged and designed in a wide variety ofdifferent configurations. Thus, the following more detailed descriptionof several configurations, as represented in the Figures, is notintended to limit scope, as claimed, but is merely representative of thesystems and methods.

FIG. 1 is a block diagram illustrating an example of one or moreelectronic devices 102 a-b in which systems and methods for reducing RPSsignal overhead may be implemented. In this example, electronic device A102 a and electronic device B 102 b are illustrated. However, it shouldbe noted that one or more of the features and functionality described inrelation to electronic device A 102 a and electronic device B 102 b maybe combined into a single electronic device in some configurations.

Electronic device A 102 a includes an encoder 104 and an RPS indexinitializer module (initializer module) 108. Each of the elementsincluded within electronic device A 102 a (e.g., the encoder 104 and theinitializer module 108) may be implemented in hardware, software or acombination of both.

Electronic device A 102 a may obtain an input picture 106. In someconfigurations, the input picture 106 may be captured on electronicdevice A 104 a using an image sensor, retrieved from memory and/orreceived from another electronic device.

The encoder 104 may encode the input picture 106 to produce encodeddata. For example, the encoder 104 may encode a series of input pictures106 (e.g., video). In one configuration, the encoder 104 may be a HEVCencoder. The encoded data may be digital data (e.g., a bitstream). Theencoder 104 may generate overhead signaling based on the input signal.

The initializer module 108 may be used in processing RPS index values.For example, the initializer module 108 may initialize values used inthe processing of positive reference pictures. This data may be recordedin a PPS and an index to it may be signaled in the slice header of apicture.

More detail on kinds of initializations that may be produced byelectronic device A 102 a is given below. It should be noted that theinitializer module 108 may be included within the encoder 104 in someconfigurations. The initializer module 108 may enable reduced RPSsignaling overhead.

The encoder 104 (and initializer module 108, for example) may produce abitstream 114. The bitstream 114 may include encoded picture data basedon the input picture 106. In some configurations, the bitstream 114 mayalso include overhead data, such as slice header information, PPSinformation, etc. More detail on overhead data is given below. Asadditional input pictures 106 are encoded, the bitstream 114 may includeone or more encoded pictures. For instance, the bitstream 114 mayinclude one or more encoded reference pictures and/or other pictures.

The bitstream 114 may be provided to a decoder 112. In one example, thebitstream 114 may be transmitted to electronic device B 102 b using awired or wireless link. In some cases, this may be done over a network,such as the Internet or a Local Area Network (LAN). As illustrated inFIG. 1, the decoder 112 may be implemented on electronic device B 102 bseparately from the encoder 104 on electronic device A 102 a. However,it should be noted that the encoder 104 and decoder 112 may beimplemented on the same electronic device in some configurations. In animplementation where the encoder 104 and decoder 112 are implemented onthe same electronic device, for instance, the bitstream 114 may beprovided over a bus to the decoder 112 or stored in memory for retrievalby the decoder 112.

The decoder 112 may be implemented in hardware, software or acombination of both. In one configuration, the decoder 112 may be a HEVCdecoder. The decoder 112 may receive (e.g., obtain) the bitstream 114.The decoder 112 may generate a decoded picture 118 (e.g., one or moredecoded pictures 118) based on the bitstream 114. The decoded picture118 may be displayed, played back, stored in memory and/or transmittedto another device, etc.

The decoder 112 may include a partial RPS determination module 120, anRPS template module 122 and/or an RPS deletion module 124. The partialRPS determination module 120 may enable the decoder 112 to generate apartial RPS at the decoder. For example, partial RPS determinationmodule 120 may generate a partial RPS based on a full RPS received atthe decoder 112. The partial RPS determination module 120 is describedin greater detail below.

The RPS template module 122 may create an RPS template based on signalsreceived in the bitstream 114. For example the RPS template module 122may use a GOP size, a coding structure and rules obtained from thebitstream to create a template. The RPS template module 122 is describedin greater detail below.

In some configurations, the RPS deletion module 124 may identify missingreference pictures and send feedback to the encoder 104, signaling thatone or more reference pictures are missing. The RPS deletion module 124may obtain instructions in the bitstream to delete one or more picturesin a current or previously received RPS. In other configurations, an RPSdeletion instructions may be received without sending feedback to theencoder. The RPS deletion module 124 is described in greater detailbelow.

In another example, such as a scalable coding scenario, a bitstream maycontain pictures of different resolutions. In this example, thereference picture collection may include (e.g., identify) differentresolution versions of the same picture.

FIG. 2 is a block diagram illustrating one configuration of a group ofpictures (GOP) 228. In some configurations, the GOP 228 may be in ahierarchy coding structure, such as Hierarchy B structure with a codingorder of minimum decoding delay. For instance, the GOP 228 may be in ahierarchical prediction structure with four dyadic hierarchy stages. Itshould be noted that the number of pictures in the GOP 228 may be eight(e.g., GOP size=8) in some configurations, as illustrated in FIG. 2.

FIG. 2 illustrates one or more pictures 226. In some instances, thesepictures may be reference pictures. Each picture 226 may belong to ahierarchy stage. For instance, I-frame pictures 226 may have a basehierarchy stage, such as stage 0. In some cases, the I-frames may be anIDR picture. B-frames may have stages of 1, 2 or 3. In some cases, thestages may correspond to temporal scalability. In other words, as thestates increase in number, so does the ability to refine the currentpicture. In this way, a picture 226 with a lower stage can employ highercorresponding pictures 226 to add clarity and refinement.

The GOP 228 may be ordered by display order or a POC 229. However, thecoding order 230 or order in which the pictures are obtained from abitstream may be different from the POC 229 order.

In some configurations, the pictures 226 in a GOP 228 are ordered in asymmetrical hierarchal pattern. In this type of configuration, thesmallest distance to a negative reference picture and a positivereference picture may be the same. That is, the absolute value ofpositive deltaPOC and negative deltaPOC in RPS are usually the same.

For example, suppose POC 2 is the current POC 231 that indicates acurrent picture to be decoded. To decode the picture indicated by thecurrent POC 231, one or more of the RPS index values 232 must beobtained. In this example, the picture indicated by the current POC 231uses pictures indicated by POC 2, POC 4 and POC 6 to be decoded. Ratherthan using absolute values, the current POC 231 uses relative valuesstored in an RPS to locate reference pictures. In some cases, the RPS isobtained directly from the slice header in a picture 226. In othercases, the RPS is located in the PPS as signaled from the slice headerof the corresponding picture 226. Here, current POC 231 obtains the RPSindex values of [−2, 2, 4]. That is, current POC 231 employs deltaPOC233 a, deltaPOC 233 b and deltaPOC 233 c to decode the picture 226.

In this example, where the current POC is POC 2, the symmetricalhierarchy nature of a GOP 228 may be beneficial in reducing the RPSsignaling overhead. The current POC 231 that has a negative deltaPOC 233a (e.g., −2) will likely have a corresponding symmetrical positivedeltaPOC 233 b (e.g., 2). Using this symmetry, approaches for reducingRPS signal overhead can be employed. Various such approaches (e.g.,methods) are described in greater detail below in connection with FIGS.5 and 6.

In another example, assuming that the currentPOC is POC 4, then one RPSindex value may be deltaPOC −4. Additionally, POC 4 may also have anindex value of deltaPOC=4 in its RPS, which refers to the referencepicture POC 8. Further RPS reference pictures for currentPOC=POC 4 maybe POC 2 and POC 6. Thus, the RPS index values may be [−4, −2, 2, 4].

In yet another example, assuming that the currentPOC is POC 2, then theRPS may include pictures corresponding to POC 0 and POC 4. Additionally,the index value deltaPOC=6 may be stored in its RPS. This is because theRPS index value of deltaPOC=6 refers to a reference picture that mayneed to be kept in the decoder since it may be used as a referencepicture by future pictures. In other words, RPS index values withpositive deltaPOC values indicating one or more pictures may need to bekept in the decoder to serve as reference pictures for a future picturethat may need to be decoded.

FIG. 3 is a block diagram illustrating one configuration of an encoder304 on an electronic device 302. It should be noted that one or more ofthe elements illustrated as included within the electronic device 302may be implemented in hardware, software or a combination of both. Forexample, the electronic device 302 includes an encoder 304, which may beimplemented in hardware, software or a combination of both. Forinstance, the encoder 304 may be implemented as a circuit, integratedcircuit, application-specific integrated circuit (ASIC), processor inelectronic communication with memory with executable instructions,firmware, field-programmable gate array (FPGA), etc., or a combinationthereof. In some configurations, the encoder 304 may be a HEVC coder.

The electronic device 302 may include a supplier 334. The supplier 334may provide picture or image data (e.g., video) as a source 306 to theencoder 304. Examples of the supplier 334 include image sensors, memory,communication interfaces, network interfaces, wireless receivers, ports,etc.

The source 306 may be provided to an intra-frame prediction module andreconstruction buffer 340. The source 306 may also be provided to amotion estimation and motion compensation module 366 and to asubtraction module 346.

The intra-frame prediction module and reconstruction buffer 340 maygenerate intra mode information 358 and an intra signal 342 based on thesource 306 and reconstructed data 380. The motion estimation and motioncompensation module 366 may generate inter mode information 368 and aninter signal 344 based on the source 306 and a reference picture buffer396 signal 398.

The reference picture buffer 396 signal 398 may include data from one ormore reference pictures stored in the reference picture buffer 396. Thereference picture buffer 396 may also include an RPS index initializermodule 308. The initializer module 308 may process reference pictures226 corresponding to the buffering and list construction of an RPS. Insome configurations, the reference picture buffer 396 signal 398 mayinclude data that indicates one or more previous reference pictures sets(RPS) corresponding to the current picture.

The encoder 304 may select between the intra signal 342 and the intersignal 344 in accordance with a mode. The intra signal 342 may be usedin order to exploit spatial characteristics within a picture in an intracoding mode. The inter signal 344 may be used in order to exploittemporal characteristics between pictures in an inter coding mode. Whilein the intra coding mode, the intra signal 342 may be provided to thesubtraction module 346 and the intra mode information 358 may beprovided to an entropy coding module 360. While in the inter codingmode, the inter signal 344 may be provided to the subtraction module 346and the inter mode information 368 may be provided to the entropy codingmodule 360.

Either the intra signal 342 or the inter signal 344 (depending on themode) is subtracted from the source 306 at the subtraction module 346 inorder to produce a prediction residual 348. The prediction residual 348is provided to a transformation module 350. The transformation module350 may compress the prediction residual 348 to produce a transformedsignal 352 that is provided to a quantization module 354. Thequantization module 354 quantizes the transformed signal 352 to producetransformed and quantized coefficients (TQCs) 356.

The TQCs 356 are provided to an entropy coding module 360 and an inversequantization module 370. The inverse quantization module 370 performsinverse quantization on the TQCs 356 to produce an inverse quantizedsignal 372 that is provided to an inverse transformation module 374. Theinverse transformation module 374 decompresses the inverse quantizedsignal 372 to produce a decompressed signal 376 that is provided to areconstruction module 378.

The reconstruction module 378 may produce reconstructed data 380 basedon the decompressed signal 376. For example, the reconstruction module378 may reconstruct (modified) pictures. The reconstructed data 380 maybe provided to a deblocking filter 382 and to the intra predictionmodule and reconstruction buffer 340. The deblocking filter 382 mayproduce a filtered signal 384 based on the reconstructed data 380.

The filtered signal 384 may be provided to a sample adaptive offset(SAO) module 386. The SAO module 386 may produce SAO information 388that is provided to the entropy coding module 360 and an SAO signal 390that is provided to an adaptive loop filter (ALF) 392. The ALF 392produces an ALF signal 394 that is provided to the reference picturebuffer 396. The ALF signal 394 may include data from one or morepictures that may be used as reference pictures.

The entropy coding module 360 may code the TQCs 356 to produce abitstream 314. Also, the entropy coding module 360 may code the TQCs 356using Context-Adaptive Variable Length Coding (CAVLC) orContext-Adaptive Binary Arithmetic Coding (CABAC). In particular, theentropy coding module 360 may code the TQCs 356 based on one or more ofintra mode information 358, inter mode information 368 and SAOinformation 388. The bitstream 314 may include coded picture data.

Quantization, involved in video compression such as HEVC, is a lossycompression technique achieved by compressing a range of values to asingle quantum value. The quantization parameter (QP) is a predefinedscaling parameter used to perform the quantization based on both thequality of reconstructed video and compression ratio. The block type isdefined in HEVC to represent the characteristics of a given block basedon the block size and its color information. QP, resolution informationand block type may be determined before entropy coding. For example, theelectronic device 302 (e.g., the encoder 304) may determine the QP,resolution information and block type, which may be provided to theentropy coding module 360.

The entropy coding module 360 may determine the block size based on ablock of TQCs 356. For example, block size may be the number of TQCs 356along one dimension of the block of TQCs. In other words, the number ofTQCs 356 in the block of TQCs may be equal to block size squared. Forinstance, block size may be determined as the square root of the numberof TQCs 356 in the block of TQCs. Resolution may be defined as a pixelwidth by a pixel height. Resolution information may include a number ofpixels for the width of a picture, for the height of a picture or both.Block size may be defined as the number of TQCs 356 along one dimensionof a 2D block of TQCs.

In some configurations, the bitstream 314 may be transmitted to anotherelectronic device. For example, the bitstream 314 may be provided to acommunication interface, network interface, wireless transmitter, port,etc. For instance, the bitstream 314 may be transmitted to anotherelectronic device via LAN, the Internet, a cellular phone base station,etc. The bitstream 314 may additionally or alternatively be stored inmemory on the electronic device 302.

FIG. 4 is a flow diagram illustrating one configuration of a method 400for reducing RPS signal overhead on an electronic device 302. Theelectronic device 302 may obtain 402 a picture 226. For example, thepicture 226 may be received from a supplier 334. In some instances, thepicture 226 may be obtained from a remote source.

The picture 226 may be encoded 404. This may be performed, for example,on an encoder 304. The encoder 304 may be an HEVC type encoder 304.

A RPS corresponding to the picture 226 based on an initialized indexvalue may be indicated 406. For example, this indication may be made inthe slice header of the picture 226 and/or in the PPS of the picture226. The initialized value may assist the electronic device 302 inprocessing the RPS. In some cases, the initialized value may help reduceRPS signal overhead. In some cases, using the initialized value mayresult in smaller RPS index values being sent in the bitstream 314, thusreducing overhead. For instance, smaller RPS index values may berepresented with fewer bits to reduce overhead. In other cases, usingthe initialized index value may result in one less RPS index value beingsent in the bitstream 314, which also reduces RPS signaling overhead.

The electronic device 302 may send 408 a bitstream. The bitstream 314may be sent 408 to the same electronic device 302 and/or a remotedevice. The bitstream 314 may include the encoded picture and an RPS.

FIG. 5 is a flow diagram illustrating a more specific configuration of amethod 500 for reducing RPS signal overhead on an electronic device 302.In one configuration, the electronic device 302 may obtain 502 a picture226 and encode 504 the picture 226 as discussed previously.

The electronic device 302 may initialize 506 an index value based on arelative negative RPS parameter. For example, the electronic device 302may process a reference picture to obtain both negative and positive RPSindex values. The positive and negative reference pictures may beprocessed individually. These processes are shown below in Listing (3).

Listing (3) illustrates syntax of one known method for coding positiveRPS index values. Specifically, Listing (3) is designed for processingRandom Access Hierarchy B under HM 5.0.

Listing (3) prev = 0; For (j=0; j < num_negative_pics; j++){     write(prev − RPS_deltaPoc(j) − 1);     prev = RPS_deltaPoc(j);     ... } prev= 0; For (j=0; j < num_positive_pics; j++){     k = num_negative_pics+j;     write(RPS_deltaPoc(k) − prev − 1);     prev = RPS_deltaPoc(k);    ... }

RPS_deltaPOC refers to the RPS index values. RPS_deltaPOC[0] toRPS_deltaPOC[num_negative_pics−1] stores the negative RPS index values.RPS_deltaPOC[num_negative_pics] toRPS_deltaPOC[num_negative_pics+num_positive_pics−1] stores the positiveRPS index values.

Listing (3) illustrates pseudo code for one approach of processingpositive RPS index values. For example, an RPS with reference indexes[−2,−4, 2], first codes the negative RPS picture indexes. Then thepositive picture indexes are differentially coded. This results in fourRPS index values. In processing both the positive and the negativereference RPS index values, the previous index value or “prev” is set to0. This is always the case when processing the positive RPS index value.prev is initialized independently of the number of negative pictures orany negative RPS parameters. In this example, when the RPS index valuesare [−2,−4, 2], 9 bits are required to send the RPS index values.

In contrast to Listing (3), the systems and methods disclosed hereindescribe that an index value may be initialized 506 based on a relativenegative RPS parameter. Thus, rather than initializing prev to 0 eachtime the positive RPS index values are processed, prev is initializedbased on a negative RPS parameter. Listing (4) illustrates an example ofpseudo code that illustrates this distinction. Modifications to thesyntax in accordance with the systems and methods disclosed herein aredenoted in bold.

Listing (4) prev = 0; For (j=0; j < num_negative_pics; j++) {  write(prev − RPS_deltaPoc(j) − 1);  prev = RPS_deltaPoc(j);  ... } prev =(smallest absolute value of negative RPS_deltaPoc) − 1; For (j=0; j <num_positive_pics; j++) {  k = num_negative_pics+ j; write(RPS_deltaPoc(k) − prev − 1);  prev = RPS_deltaPoc(k);  ... }

As illustrated in Listing (4), prev may be initialized to the smallestabsolute value of negative RPS_deltaPoc minus 1. It should be noted thatthe terms RPS_deltaPOC and RPS index values may be used interchangeably.The smallest absolute value of negative RPS_deltaPoc is the absolutevalue of the smallest negative reference picture stored in the RPS indexfor a current picture. For example, if a current picture contains therelative reference RPS index values of [−4, −6, 4], −4 and −6 are thenegative RPS_deltaPoc index values. When the absolute values of each aretaken, 4 and 6 are obtained, respectively. Taking the smallest valueresults in obtaining 4.

In a known approach, RPS_deltaPOC values are arranged in ascending orderby absolute value within the negative RPS_deltaPOC index values andwithin the positive RPS_deltaPOC index values to ensure that thedifference values coded are not negative. This allows for less bits whencoding because negative numbers are avoided. In general, a negativenumber takes more bits to code than a positive bit that is the absolutevalue of the same negative number. Under this approach, the smallestabsolute value of negative RPS_deltaPoc may be located atRPS_deltaPoc(0).

Applying this example, prev=(smallest absolute value of negativeRPS_deltaPoc)−1, prev=(4)−1=3. In another example where the smallestabsolute value of negative of RPS_deltaPoc is 5, prev would equal 4.Thus, the prev index value may be dependent on a negative RPS parametervalue such as the smallest absolute value of negative of RPS_deltaPoc.

Using the initialized value, the electronic device 302 may process 508 apositive number of pictures based on the index value. In one example,suppose a current picture has an RPS reference index values of [−2, −4,2]. When the positive reference pictures are processed and prev isinitialized to the smallest absolute value of negative of RPS_deltaPoc,prev equals 1 (e.g., (|−2|)−1=1). To process the first positive value(e.g., 2), RPS_deltaPoc(k)−prev−1 is employed. RPS_deltaPoc(k) equalsthe value of the first positive RPS index value to be processed. In thisexample, RPS_deltaPoc(k) equals 2. prev equals 1, as shown above. Thus,RPS_deltaPoc(k)−prev−1=2−1 −1=0.

This processing results in a coding of [−2, −4, 0] rather than [−2, −4,2]. Thus, 8 bits rather than 9 bits are required and overhead isreduced. Due to the symmetrical nature of the coding structure,initializing the index value based on the relative negative RPSparameter will generally result in the first positive index referencevalue equaling zero. Thus, as the first positive index reference valueincreases, the number of bits saved also increases.

In some instances, the electronic device 302 may process only the firstpositive RPS index value in the set of RPS index values based on therelative negative RPS parameter. Each subsequent positive index valuemay be processed, as shown in Listing (3). In this case, overhead isreduced for the first positive index value for each RPS of each picturesent in the bitstream 114.

The electronic device 302 may send 510 a bitstream 114. By applying thesystems and methods described herein, the RPS overhead in the bitstream114 may be reduced.

FIG. 6 is a flow diagram illustrating another more specificconfiguration of a method 600 for reducing RPS signal overhead on anelectronic device 302. The electronic device 302 may obtain 602 apicture 226 and encode 604 the picture 226, as discussed previously.

The electronic device may optionally indicate 606 a symmetric flag, suchas symmetric_flag, to be sent in bitstream to indicate if the firstpositive RPS index value can be initialized with a negative RPS indexvalue. For example, the symmetric flag may indicate that the index valueshould be initialized to the smallest absolute value of negativeRPS_deltaPoc. Alternatively, the symmetry flag may indicate notinitializing the index value to the smallest absolute value of negativeRPS_deltaPoc. In this case the index value may be initialized to zero.

The electronic device 302 may process 608 a positive number of picturesbased on the index value. In some cases, the processing skips when theindex value is initialized. In other words, when the index value isinitialized, the first iteration of processing the positive RPS indexvalues skips writing the first positive RPS index value to the RPS. Inthe case where only one positive number exits, then no processing ofpositive numbers occur. The electronic device 302 may also send 610 abitstream 314.

Listing (5) shows one example of pseudo code syntax in which the presentmethod may be applied. Modifications to the syntax in accordance withthe systems and methods disclosed herein are denoted in bold.

Listing (5) prev = 0; For (j=0; j < num_negative_pics; j++){   write(prev − RPS_deltaPoc(j) − 1);   prev = RPS_deltaPoc(j);   ... } prev = 0;if (num_positive_pics)   write(symmetric_flag); For (j=0; j <num_positive_pics; j++){   k = num_negative_picsnumNegativePictures+ j;  if (j>0 || ! symmetric_flag)     write(RPS_deltaPoc(j) − prev − 1);  prev = RPS_deltaPoc(k);   ... }

When j>0 in processing the positive RPS index value, the condition j==0is skipped. In other words, the first positive reference picture isskipped.

Returning now to the condition where j>0. In this instance, j isinitialized to 0 in the for loop when processing the positive RPS indexvalues. Upon initialization, the processing of positive RPS index valuesskips the first positive RPS index value. For example, in the case ofRPS index values [−8, −4, 4, 6], in processing the positive RPS indexvalues, 4 will be skipped over and only 6 will be added to the RPS indexfor the current picture.

In another example, for index values [−2, −4, 2, 6], if thesymmtetic_flag is true, for the positive RPS_deltaPOC values, the firstone must be equal to 2, as derived from the negative reference pictureindexes. In this manner, processing of the positive RPS index values maybegin from the second positive RPS index value (e.g., 6). Hierarchicalcoding structures, such as Random Access (Hierarchical B), the symmetriccondition between the start of positive and negative RPS index isusually true. This is due to the symmetrical nature of each currentpicture in relation to deltaPOC reference pictures.

In the case of only one positive RPS index value, the electronic device302 may skip processing of the positive values. For example, for indexvalues [−4, −8, 4], after processing, an RPS of [−4, −8] may be sent tothe bitstream. Then upon receipt, the receiving electronic device, suchas a decoder, may receive the RPS index values [−4, −8] and derivereference index values of [−4, −8, 4] for the current picture beingdecoded. Thus, by skipping the first positive reference picture in eachRPS for each current picture, significant overhead savings may beachieved in RPS signaling.

For example, Listing (6) shows syntax for processing RPS index values ona decoder.

Listing (6) ref_pic_set( idx ) {   num_negative_pics   num_positive_pics  If (num_positive_pics) {     symmetric_flag   }   for( i = 0; i <num_negative_pics; i++ ) {     delta_poc_s0_minus1[ i ]    used_by_curr_pic_s0_flag[ i ]   }   for( i = 0; i <num_positive_pics; i++ ) {     If (i!=0 || ! symmetric_flag) {      delta_poc_s1_minus1[ i ]   }   used_by_curr_pic_s1_flag[ i ]   } }

Listing (6) shows one example a syntax which can be processed by adecoder. This syntax may be generated by an electronic device 302 usingthe approach discussed in connection with Listing (5). For example, thedecoder may check symmertric_flag to see if there is an indication thatthe smallest negative RPS index value is symmetrical with acorresponding positive RPS index value. If symmertric_flag indicatessuch, the decode may decode the RPS and use the absolute value of thesmallest negative RPS index value at the RPS index value of the firstpositive RPS index value. It will be appreciated that other approachesto decode a picture as listed in Listing (6) may be employed whichcorrespond to the encoding approach discussed above in connecting withListing (5).

FIG. 7 is a block diagram illustrating one configuration of a decoder712 on an electronic device 702. The decoder 712 may be included in anelectronic device 702. For example, the decoder 712 may be a HEVCdecoder. The decoder 712 and/or one or more of the elements illustratedas included in the decoder 712 may be implemented in hardware, softwareor a combination of both. The decoder 712 may receive a bitstream 714(e.g., one or more encoded pictures included in the bitstream 714) fordecoding. In some configurations, the received bitstream 714 may includereceived overhead information, such as a received slice header, receivedPPS, received buffer description information, etc. The encoded picturesincluded in the bitstream 714 may include one or more encoded referencepictures and/or one or more other encoded pictures.

Received symbols (in the one or more encoded pictures included in thebitstream 714) may be entropy decoded by an entropy decoding module 768,thereby producing a motion information signal 770 and quantized, scaledand/or transformed coefficients 772.

The motion information signal 770 may be combined with a portion of areference frame signal 798 from a frame memory 778 at a motioncompensation module 774, which may produce an inter-frame predictionsignal 782. The quantized, descaled and/or transformed coefficients 772may be inverse quantized, scaled and inverse transformed by an inversemodule 762, thereby producing a decoded residual signal 784. The decodedresidual signal 784 may be added to a prediction signal 792 to produce acombined signal 786. The prediction signal 792 may be a signal selectedfrom either the inter-frame prediction signal 782 or an intra-frameprediction signal 790 produced by an intra-frame prediction module 788.In some configurations, this signal selection may be based on (e.g.,controlled by) the bitstream 714.

The intra-frame prediction signal 790 may be predicted from previouslydecoded information from the combined signal 792 (in the current frame,for example). The combined signal 792 may also be filtered by ade-blocking filter 794. The resulting filtered signal 796 may be writtento frame memory 778. The resulting filtered signal 796 may include adecoded picture.

The frame memory 778 may include a DPB as described herein. The DPB mayinclude one or more decoded pictures that may be maintained as short orlong term reference frames. The frame memory 778 may also includeoverhead information corresponding to the decoded pictures. For example,the frame memory 778 may include slice headers, PPS information, cycleparameters, buffer description information, etc. One or more of thesepieces of information may be signaled from an encoder (e.g., encoder304). The frame memory 778 may provide a decoded picture 718.

The decoder 712 may include a partial RPS determination module 720, anRPS template module 722 and/or an RPS deletion module 724. The partialRPS determination module 720 may generate a partial RPS based on signalsobtained from the bitstream 714. In some cases, a partial RPS may bedetermined on the decoder 712 based on a full RPS. Greater detailregarding partial RPS determination module 720 is described below.

The RPS template module 722 may derive an RPS template at the decoder712. In some instances, the RPS template may be created based on signalsreceived for the bitstream 714, such as a GOP size, a coding structureand rules.

The RPS deletion module 724 may receive indications at the decoder 712to delete a previously received RPS. For example, the bitstream 714 mayinclude a flag in a PPS and a bit field in a slice header of a picture.The RPS deletion module 724 may assist in identifying missing referencepictures. Greater detail regarding the RPS deletion module 724 will begiven below.

FIG. 8 is a flow diagram illustrating one configuration of method 800for reducing RPS signal overhead on an electronic device 702. Theelectronic device 702 may obtain 802 a bitstream 714. For example, thebitstream 714 may be obtained from the electronic device 702 where adecoder 712 is located, or alternatively from another electronic device.

The electronic device 702 may obtain 804 a GOP based on the bitstream714. In some cases, a GOP may be specified as a specific number ofsequential pictures, such as four or eight pictures. For example, theelectronic device 702 may obtain 804 a GOP that includes 8 pictures.

There may be various ways that the electronic device 702 may derive afull RPS. In one configuration, the electronic device 702 may receiveGOP size, coding structure and other necessary information, such as thePPS.

To derive the full RPS, the electronic device 702 may determine thecoding order according to coding structure and GOP size. For instance,with low delay coding, the coding order may be the display order. Inanother instance, with Hierarchical B, the coding order may employminimum decoding delay. In one example that is similar to the exampleillustrated in FIG. 2, a GOP, with a size of 8, will code in the orderof POC 8, 4, 2, 1, 3, 6, 5, 7. Alternatively, a coding order may becoded from a lower hierarchy towards higher hierarchy. In this example,a GOP, with the size of 8, will code in the order of POC 0, 8, 4, 2, 6,1, 3, 5, 7. Additionally, a flag can be used to denote which codingorder to use. In some cases, the flag may be a bit flag.

To derive the full RPS, the electronic device 702 may also determine thereference pictures based on coding structure, temporal layer id, numberof reference pictures and additional rules set by the encoder and thedecoder. For example, in the case of low delay coding with N referenceframe, a preceding picture with same or lower temporal layer may alwaysserve as a reference picture. Then, for the remaining N−1 referencepictures, additional rules can be specified to select reference picturesdepending on coding order and/or hierarchy layer. A parameter may beused to specify which rule should be used. An example of this parameteris reference_picture_selection rule, and is discussed below inassociation with Listing (9).

Additionally, to derive the full RPS, the electronic device 702 may loop(e.g., iterate) the GOP in coding order. The electronic device 702 mayadd or keep reference pictures that will be later referenced inconnection with subsequent pictures. These subsequent pictures may be inthe same or later GOPs then the current picture. If a reference picturesis to be used for reference later, it may be marked for future use.Otherwise, it may be marked, “not for reference.”

The electronic device 702 may derive 806 a partial RPS from a full RPSbased on at least one relative index value. For example, the electronicdevice 702 may attempt to derive a full RPS.

However, because some reference pictures are missing or have not beenobtained, only a partial RPS may be derived.

A partial RPS may be for frames that have unavailable reference picturesdue to IDR or CRA, such as occurs at the start of a picture sequence.For example, suppose a first picture is at POC 8 with RPS index valuesof [−8, −10, −12, −16]. Being the first received picture, it does notyet have reference pictures corresponding to RPS index values [−10, −12,−16]. In this example, the first picture cannot use the full RPS withindex reference values of [−8, −10, −12, −16]. Rather, a partial RPScontains index value [−8], and this index value represents the onlyreference picture that may be used to decode the picture (e.g., POC 8).

The electronic device 702 may then decode 808 a picture based on thepartial RPS. In other words, the electronic device 702 may decode 808 apicture using the reference pictures available to it.

In some implementations, a partial RPS may be additionally derived onthe encoder 304 side of an electronic device 302. However, by deriving apartial RPS at the decoder 712, RPS overhead in the bitstream 714 isreduced due to the partial RPS not being sent over in addition to thefull RPS being sent over.

Listing (7) shows one approach for deriving a partial RPS.

Listing (7)   pic_parameter_set_rbsp( ) {   pic_parameter_set_id  seq_parameter_set_id   entropy_coding_mode_flag  num_full_ref_pic_sets   num_partial_ref_pic_sets   for(idx = 0; idx <num_full_ref_pic_sets; idx++)    ref_pic_set( idx )   i=0   while( idx <NumRefPicSets ) {    partial_ref_pic_set_flag[ i ]    if(partial_ref_pic_set_flag ) {      for( j=0; j < NumNegativePics[ i %num_full_ref_pic_set ];      j++ )        ref_flag[ j ]      idx++    }  i++   } }

In Listing (7), num_full_ref_pic_sets specifies the number of fullreference picture sets that are specified in the picture parameter set.num_partial_ref_pic_sets specifies the number of partial referencepicture sets that are specified in the picture parameter set.ref_flag[j] specifies whether the content of the full reference pictureset is copied to the partial reference picture set.

FIG. 9 is a flow diagram illustrating one configuration of a method 900for deriving a partial RPS on an electronic device 702. In order todecode a picture, the electronic device 702 may need to reference theRPS of the current picture. However, the full RPS may not be available.In this case, a partial RPS may be constructed.

In one configuration, a partial RPS may be constructed by the electronicdevice 702. The electronic device 702 may obtain 902 a bitstream 714 andobtain 904 a GOP based on the bitstream in a similar manner, asdescribed above.

The electronic device 702 may iterate 906 over each picture within theGOP. For instance, the electronic device 702 may loop through eachpicture and identify the RPS index values corresponding to each picture.In some cases, iteration may also occur over multiple GOPs as well.

The electronic device 702 may identify 908 a current picture, where therelative index value corresponds to the current picture. For example,this may be the picture the electronic device 702 is trying to decode.Additionally, in identifying 908 a current picture, the electronicdevice 702 may also identify the POC for the current picture. Forinstance, the POC may specify the POC of the current picture in the GOP.

Listing 8 below shows and example of syntax for generating a partialRPS. In Listing (8) current_poc_in_GOP specifies the current POC in aGOP.

Listing (8) current_poc_in_GOP ranges from 1 to the GOP size.  pic_parameter_set ( ) {    pic_parameter_set_id   seq_paramater_set_id    entropy_coding_mode_flag    num_ref_pic_sets   for(idx=0; idx < num_ref_pic_sets_idx++){      ref_pic_set(idx)     current_poc_in_GOP    }    ...   }   ref_pic_set(idx){   num_negative_pics    num_positive_pics    for(i = 0; i <num_negative_pics; i++){      delta_poc_s0_minus1[i]     used_by_curr_pic_s0_flag[i]    }    for(i = 0; i <num_positive_pics; i++){      delta_poc_s1_minus1[i]     used_by_curr_pic_s1_flag[i]   } }

The electronic device 702 may iterate 910 over the at least one relativeindex value within the full RPS of the current picture. For example, ifthe current picture has RPS index values of [−1,−5,−9,−13], then theelectronic device would loop through reference pictures located atdeltaPOC=−1, deltaPOC=−5, deltaPOC=−9 and deltaPOC=−13.

The electronic device 702 may determine 912 whether a reference pictureis available for each relative index value. In the example above, theelectronic device 702 may attempt to access reference pictures locatedat deltaPOCs=−1, −5, −9 and −13. However, one or more the referencepictures may be missing.

One approach for determining 912 whether a reference picture isavailable is by satisfying the condition currentPOC+refDeltaPOC<POC_of_Latest_CRA. POC_of_Latest_CRA indicates the position ofthe latest clean random access (CRA) picture. In other words,POC_of_Latest_CRA is the POC of the latest IDR or CRA picture. If an RPSindex values refers to a reference picture located before the latest CRApicture, then the reference picture will not be available. Thus, ifcurrentPOC+ref DeltaPOC<POC_of_Latest_CRA is satisfied, then therelative reference picture is missing and is not added to the partialRPS.

As an example, suppose the current picture has a POC of 1. (e.g., POC=1)and POC_of_Latest_CRA is 0. Again, suppose that the current picture hasRPS index values of [−1,−5,−9,−13]. Applying the conditioncurrentPOC+ref DeltaPOC for the first index value results in 0 (e.g.,1+−1=0). Because the condition is not satisfied (e.g., 1+−1 is not lessthan 0 or POC_of_Latest_CRA), the first reference index value is addedto the partial RPS. Applying currentPOC+ref DeltaPOC to the other RPSreference index values produce results where currentPOC+ref DeltaPOC isless than POC_of_Latest_CRA. Thus, no other index values are added tothe partial RPS.

As another example, suppose the current picture has a POC of 5. (e.g.,POC=5). Again, suppose that the current picture has RPS index values of[−1,−5,−9,−13]. Applying the condition currentPOC+ref DeltaPOC for theRPS index values results in [4, 0, −4, −8], respectively. Because thefirst two values are not less than zero, the corresponding RPS referenceindex values are added to the partial RPS. Thus, the partial RPS createdin this example has the index values of [−1, −5].

The electronic device 702 may add 914 the reference picture to thepartial RPS if the picture is available. The electronic device 702 mayalso decode 916 the current picture using the partial RPS index values.

In yet other configurations, the electronic device 702 may evaluate thepartial RPS and determine to insert additional pictures. For example, apartial RPS with only one reference to a reference picture may bemodified to include three additional reference pictures that may beuseful in decoding that current picture.

FIG. 10 is a flow diagram illustrating one configuration of a method1000 for deriving an RPS template on an electronic device 702. Asdiscussed in connection with Tables (1)-(4), RPS templates may be usedin video coding to assist in processing frequently used codingstructures. An example of such structure is a Random Access HierarchicalB coding structure with a GOP size equal to 8 or 16. Another example isa Low Delay coding structure with the GOP size equal to 4 or 8.

In one configuration, an RPS template may be created. It should be notedthat creation on an RPS template may be derived at both the encoder anddecoder side without sending the full RPS in the PPS. Further,derivation of the RPS template may occur in connection with, orindependent or the creation of a partial RPS, as discussed above.

The electronic device 702 may obtain 1002 a bitstream 714. Theelectronic device 702 may obtain 1004 a flag that indicates creating theRPS template. For example, the flag may be located and sent in the PPS.

As another example, the electronic device 702 may have and/or obtain aset of rules used to derive RPS templates for commonly used GOP sizes,coding structures, temporal layer settings and temporal interleavingpatterns.

Using these rules, the electronic device 702 may create 1006 an RPStemplate based on a reduced amount of information. For instance, an RPStemplate may be derived by using information regarding the GOP size, thecoding structure and rules. The coding structure, for instance, mayspecify Random Access (Hierarchical B) or Low Delay. The rules may, forexample, specify the settings for the temporal layers. Listing (9) belowshows

Listing (9) below shows syntax of an approach for deriving an RPS.

Listing (9) Picture_parameter_set( idx ) { ... derive_rps_with_model_flag  If (derive_rps_with_model){  coding_structure   GOP_size_log2   for (i=0; i < GOP_size_log2; i++) {   number_reference_pictures   }   reference_picture_selection_rule   if(coding_structure==0) {    coding_order   }   temporal_layer_exist_flag  If (temporal_layer_exist_flag)    temporal_id_ordering  } } ...

derive_rps_with_model_flag specifies whether to derive an RPS at thedecoder side. derive_rps_with_model_flag set to 0 indicates to send theRPS in the PPS. derive_rps_with_model_flag set to 1 indicates to derivethe RPS at the decoder.

coding_structure equal to 0 means that the coding structure isHierarchical B coding. A coding structure equal to 1 denotes that thecoding structure is Low delay coding.

GOP_size_log 2 refers to the log 2 of GOP size. GOP size equals 2^((GOP)^(—) ^(size) ^(—) ^(log2)). number_of_reference_pictures(i) specifiesthe number of reference pictures used per reference picture list (e.g.,Ref PicList0 and/or RefPicList1) for the i^(th) hierarchical layer.reference_picture_selection_rule specifies the commonly defined rules toselect a reference picture by an encoder and a decoder.

coding_order may specify rules to determine coding order. A coding_orderof 0 may denote a first coding order and a coding_order of 1 may denotea second coding order. In one configuration, for example, a coding_orderof 0 may denote the minimum decoding delay order and a coding_order of 1may denote a hierarchy priority order (e.g., coding from the lowesthierarchy layer to a higher hierarchy layer).

temporal_layer_exist_flag specifies if different temporal layers exist.A flag equal to 0 means all frames have the same temporal layer and aflag equal to 1 means different temporal layers exist.

temporal_id_ordering specifies the ordering of pictures with sametemporal_id in a current GOP. For instance, a value of 1 indicates thatpictures in the current GOP with same temporal_id are continuous indecoding order. Alternatively, a value of 0 indicates that pictures inthe current GOP with the same temporal_id are interleaved in decodingorder.

Using the created RPS template, the electronic device 702 may decode1008 a picture. In the case of non-common RPS code structures, the RPStemplate may also be signaled directly sending the RPS template in PPSand/or slice header.

FIG. 11 is a flow diagram illustrating one configuration of a method1100 for creating an indication to delete at least one reference pictureon an electronic device 302. In some known implementations, such as thatgiven in Joint Collaborative Team on Video Coding (JCTVC) documentJCTVC-F803_d5, indicating a delete operation on a previously signaledRPS is not supported. In other words, the PPS does not support sending asignal to delete a previously sent RPS.

In another known implementation, such as JCTVC-G637, pruning ofreference picture sets have been proposed. In this implementation, anumber of short-term reference pictures that are omitted from aparticular RPS may be signaled in the PPS for pruning. One example ofwhere some of the pictures signaled in RPS are not available, is thefirst GOP following a random access point. However, JCTVC-G637 onlyallows for pruning consecutive negative pictures.

In yet another known implementation, such as JCT-G198, a signalingscheme is proposed that sends a constant deltaRPS value. Also, a sentfield value (e.g., 00) may signal that a previous reference pictureshould not be used (e.g., should be skipped).

In some known implementations, long-term reference pictures (LTRPs) areexplicitly signaled individually in each slice header. Thus, when a LTRPis not signaled, it will be deleted. Accordingly, sending a deletesignal may not be required for a LTRP. However, in some implementationswhere a LTRPs are signaled, the delete mechanism disclosed herein mayadditionally be applied to a LTRP.

The systems and methods disclosed herein may provide one or moreadditional benefits in allowing for signaling a delete operation on apreviously signaled RPS. Additional benefits may be obtained by alsoefficiently signaling a new RPS. In one configuration, an electronicdevice 302 optionally receives 1102 feedback indication of at least onemissing reference picture. The feedback may be from the same or adifferent electronic device. For example, a decoder 712 decoding apicture may discover a missing reference picture in an RPS and sendfeedback to the electronic device 302 regarding the missing referencepicture.

In another configuration no feedback is received by 302. Instead, theelectronic device 302 may send the delete indication after a new CRA orIDR picture has been signaled. For example, a CRA or IDR picture maytrigger the electronic device to send a delete indication to delete oneor more unnecessary reference pictures listed in a preciously receivedRPS.

The electronic device 302 may create 1104 an indication to delete atleast one reference picture listed in a previously provided RPS. Thisstep may occur in conjunction with, or independent of receiving feedbackindicating a missing picture. The electronic device 302 may send 1106the indication in a bitstream.

As an example, a delete indication (e.g., delete signal) may be sentafter the first GOP following a random access point. This may be donewithout the need to receive a feedback from the decoder 712. The deleteindication may be sent as a flag in PPS. For example, the flag value of1 is sent in the PPS to indicate existence of a deletion bit field inthe slice header. Alternatively, the flag value of 0 is sent in the PPSto indicate a deletion bit field is not sent in the slice header.

Additionally and/or alternatively, the delete indication may be sent asa bit field in the slice header for each picture in an indicated RPS.For example, the bit field may send the indication value of 0 in theslice header to indicate deletion of a corresponding reference picturefrom the indicated RPS or the bit field may send the indication value of1 in the slice header to indicate that the corresponding referencepicture should be kept. The bit field may be sent in the slice headerfor each of the reference pictures in the indicated RPS. In anotherconfiguration, such a bit field may be sent in the slice header only forthe negative reference pictures in the indicated RPS. In someconfigurations, the delete indication may be sent both in the PPS and inthe bit field.

Listing (10) below provides one example of syntax that may be used inthe PPS to send a delete indication. Modifications to the syntax inaccordance with the systems and methods disclosed herein are denoted inbold.

Listing (10) /* Picture parameter set raw byte sequence payload (RBSP)syntax */ pic_parameter_set_rbsp( ) {   pic_parameter_set_id  seq_parameter_set_id   ...   delete_pics_info_present_flag   ...

In Listing (10), a delete_pics_info_present_flag equal to 1 specifiesthat the slice header may signal a delete operation from the short-termreference picture set included in the picture parameter set. Adelete_pics_info_present_flag equal to 0 specifies that no deleteoperation is signaled in the slice header for the short-term referencepicture set included in the picture parameter set. pic_parameter_set_idand seq_parameter_set_id are defined as described above.

Listing (11) below provides one example of syntax that may be used inthe slice header to send a delete indication. Modifications to thesyntax in accordance with the systems and methods disclosed herein aredenoted in bold.

Listing (11) slice_header( ) {  ...  pic_order_cnt_lsb short_term_ref_pic_set_pps_flag  if( !short_term_ref_pic_set_pps_flag )  short_term_ref_pic_set( num_short_term_ref_pic_sets )  else {  short_term_ref_pic_set_idx   if(delete_pics_info_present_flag )  for(j=0;j<(num_negative_pics+num_positive_pics);j++) {   keep_pic_flag[j]   }  } ...

In Listing (11), keep_pic_flag[j] equal to 1 specifies that thecorresponding j^(th) short-term reference picture from the referencepicture set in the picture parameter set with indexshort_term_ref_pic_set_idx should be kept. keep_pic_flag[j] equal to 0specifies that the corresponding j^(th) short-term reference picturefrom the reference picture set in the picture parameter set with indexshort_term_ref_pic_set_idx should be deleted (e.g., omitted).

pic_order_cnt_lsb specifies the picture order count moduloMaxPicOrderCntLsb for the top field of a coded frame or for a codedfield. The length of the pic_order_cnt_lsb syntax element is log2_max_pic_order_cnt_lsb_minus4+4 bits. The value of thepic_order_cnt_lsb shall be in the range of 0 to MaxPicOrderCntLsb−1,inclusive. MaxPicOrderCntLsb refers to a maximum possible value for(pic_order_cnt_lsb+1). log 2_max_pic_order_cnt_lsb_minus4 specifies thevalue of the variable MaxPicOrderCntLsb that is used in the decodingprocess for picture order count as follows: MaxPicOrderCntLsb=2^((log2)^(—) ^(max) ^(—) ^(pic) ^(—) ^(order) ^(—) ^(cnt) ^(—) ^(lsb) ^(—)^(minus4+4)). The value of log 2_max_pic_order_cnt_lsb_minus4 may be inthe range of 0 to 12, inclusive.

short_term_ref_pic_set_pps_flag equal to 1 specifies that the short-termreference picture set of the current picture shall be created usingsyntax elements in the active picture parameter set.short_term_ref_pic_set_pps_flag equal to 0 specifies that the short-termreference picture set of the current picture shall be created usingsyntax elements in the short_term_ref_pic_set( ) syntax structure in theslice header.

short_term_ref_pic_set_idx specifies the index to the list of theshort-term reference picture sets specified in the active pictureparameter set that shall be used for creation of the reference pictureset of the current picture. The syntax elementshort_term_ref_pic_set_idx shall be represented by ceil(log2(num_short_term_ref_pic_sets)) bits. The value ofshort_term_ref_pic_set_idx shall be in the range of 0 tonum_short_term_ref_pic_sets−1, inclusive, wherenum_short_term_ref_pic_sets is the syntax element from the activepicture parameter set.

The variable StRpsIdx is derived as shown in Listing (12) below.

Listing (12) if( short_term_ref_pic_set_pps_flag )   StRpsIdx =short_term_ref_pic_set_idx else    StRpsIdx =num_short_term_ref_pic_sets

In Listing (12), short_term_ref_pic_set indicates a short-term referencepicture set and num_short_term_ref_pic_sets indicates the total numberof short-term reference picture sets.

FIG. 12 is a flow diagram illustrating one configuration of a method1200 for deleting at least one reference picture on an electronic device702. The electronic device 702 may optionally detect 1202 at least onemissing reference picture in the RPS. For example, the electronic device702 may perform steps similar to those described above in connectionwith creating a partial RPS, in which a determination is made as towhether a reference picture is available. Additionally or alternatively,the electronic device 702 may call a reference picture and discover thatit is missing or is corrupt or may detect 1202 a missing referencepicture in other ways.

The electronic device 702 may optionally send 1204 feedback indicatingthe at least one missing reference picture. For example, feedbackthrough a back channel indicating the missing POC, RPS or a variety ofother signals may be sent to indicate a missing picture.

The electronic device 702 may obtain 1206 an indication to delete atleast one reference picture listed in a previously obtained RPS. In oneinstance, the signal may be obtained from a flag in PPS and a bit fieldin slice header for each picture in RPS. For example,delete_pics_info_present_flag and keep_pic_flag[j] may be signaled.

The electronic device 702 may delete 1208 the at least one referencepicture in the previously obtained RPS. In some cases, the deleteindication may be received with a subsequent RPS that should replace theRPS to be deleted. In another case, the indication may specify deletinga previously received RPS that is no longer needed to decode furtherpictures. In yet another case, the indication may be to delete apreviously received RPS so that a subsequent replacement RPS may betransmitted. It should be noted that the steps of obtaining 1206 anindication to delete at least one reference picture and deleting 1208the at least one reference picture may be done in conjunction with, orindependent of the previous steps of detecting 1202 a missing pictureand sending 1204 feedback.

As an example of this, if a previously obtained RPS had the index valuesof [−2, −1, 1, 3, 5], and the electronic device 702 receives anindication to delete the second reference picture, then the previouslyobtained reference set would delete the RPS index value of −1. In otherwords, the previously obtained reference set would become [−2, 1, 3, 5].

FIG. 13 illustrates various components that may be utilized in anelectronic device 1302. The electronic device 1302 may be implemented asone or more of the electronic devices (e.g., electronic devices 102,302, 702) described herein.

The electronic device 1302 includes a processor 1317 that controlsoperation of the electronic device 1302. The processor 1317 may also bereferred to as a CPU. Memory 1311, which may include both read-onlymemory (ROM), random access memory (RAM) or any type of device that maystore information, provides instructions 1313 a (e.g., executableinstructions) and data 1315 a to the processor 1317. A portion of thememory 1311 may also include non-volatile random access memory (NVRAM).The memory 1311 may be in electronic communication with the processor1317.

Instructions 1313 b and data 1315 b may also reside in the processor1317. Instructions 1313 b and/or data 1315 b loaded into the processor1317 may also include instructions 1313 a and/or data 1315 a from memory1311 that were loaded for execution or processing by the processor 1317.The instructions 1313 b may be executed by the processor 1317 toimplement the systems and methods disclosed herein.

The electronic device 1302 may include one or more communicationinterface 1319 for communicating with other electronic devices. Thecommunication interfaces 1319 may be based on wired communicationtechnology, wireless communication technology, or both. Examples of acommunication interface 1319 include a serial port, a parallel port, aUniversal Serial Bus (USB), an Ethernet adapter, an IEEE 1394 businterface, a small computer system interface (SCSI) bus interface, aninfrared (IR) communication port, a Bluetooth wireless communicationadapter, a wireless transceiver in accordance with 3rd GenerationPartnership Project (3GPP) specifications and so forth.

The electronic device 1302 may include one or more output devices 1323and one or more input devices 1321. Examples of output devices 1323include a speaker, printer, etc. One type of output device that may beincluded in an electronic device 1302 is a display device 1325. Displaydevices 1325 used with configurations disclosed herein may utilize anysuitable image projection technology, such as a cathode ray tube (CRT),liquid crystal display (LCD), light-emitting diode (LED), gas plasma,electroluminescence or the like. A display controller 1327 may beprovided for converting data stored in the memory 1311 into text,graphics, and/or moving images (as appropriate) shown on the display1325. Examples of input devices 1321 include a keyboard, mouse,microphone, remote control device, button, joystick, trackball,touchpad, touchscreen, lightpen, etc.

The various components of the electronic device 1302 are coupledtogether by a bus system 1329, which may include a power bus, a controlsignal bus and a status signal bus, in addition to a data bus. However,for the sake of clarity, the various buses are illustrated in FIG. 13 asthe bus system 1329. The electronic device 1302 illustrated in FIG. 13is a functional block diagram rather than a listing of specificcomponents.

FIG. 14 is a flow diagram illustrating one configuration of a method1400 for decoding a picture on an electronic device 702. The electronicdevice 702 may obtain 1402 a bitstream 714. For example, the bitstream714 may be obtained from the electronic device 702 (from memory, forexample), or may be received from another electronic device.

The electronic device 702 may obtain 1404 a current picture. Forinstance, the electronic device 702 may obtain 1404 a current pictureembedded in the bitstream 714.

The electronic device 702 may obtain 1406 a relative RPS parameter. Forexample, the relative RPS may be signaled in the PPS. Accordingly, therelative RPS parameter may be obtained from the PPS. In someconfigurations, the relative RPS parameter may be a negative relativeRPS parameter. In other configurations, the relative RPS parameter maybe a positive relative RPS parameter. In some configurations, therelative negative RPS parameter may be a negative RPS index value. Forexample, a negative RPS index value may be relative to the currentpicture obtained. In other words, the relative RPS index value may referto the location of a reference picture relative to the index position ofthe current picture.

While some examples and configurations refer to the relative RPSparameter as a negative RPS parameter, it should be appreciated that apositive parameter may be used in a respective manner. In other words,even though examples are given herein pertaining to a negative relativeRPS parameter, similar examples using a positive relative RPS parametermay be employed. Likewise, other examples to positive or negativeconnotation, in many cases, may be reversed.

The electronic device 702 may initialize 1408 an index value. In someconfigurations, the electronic device 702 may initialize the index valueto 0. In other configurations, the electronic device 702 may initializethe index value based on an obtained parameter.

The electronic device 702 may process 1410 another RPS parameter basedon the index value. In some configurations, the other RPS parameter maybe a positive RPS parameter. In other configurations, the other RPSparameter may be a negative RPS parameter. In some configurations, theother RPS parameter may be a positive RPS index value. For example,Listing (13) below illustrates one approach to processing a positive RPSparameter based on the index value.

Listing (13) prev = 0; For (j=0; j < num_negative_pics; j++){   value =read( ) ;    RPS_deltaPoc(j) = prev − value −1 ;    prev =RPS_deltaPoc(j);    ... } prev = 0; For (j=0; j < num_positive_pics;j++){    k = num_negative_pics+ j;    value = read( ) ;   RPS_deltaPoc(k) = value + 1 + prev ;   prev = RPS_deltaPoc(k);    ...}

In Listing (13), value=read( ) represents reading information from thebitstream. In some configurations, the index value may be “prev” asshown in Listing (13). In some configurations, the index value may beRPS_deltaPoc. Furthermore, in some configurations, the index value maybe initialized to 0. In other configurations, the index value may beinitialized to the smallest absolute value of the negative RPS indexvalue. In yet other configurations, the index value may be initializedto the negative of the smallest value of a positive RPS index value(e.g., the smallest positive RPS index value times −1). In someconfigurations or instances, the index value may be initialized to anon-zero value.

Similar to the relative RPS parameter, it should be noted that the otherRPS parameter may be a positive or negative. In other words, even thoughexamples are given herein pertaining to a positive RPS parameter,similar examples using a negative RPS parameter may be employed.Accordingly, the relative RPS parameter may be negative while the otherRPS parameter is positive in some configurations. Alternatively, therelative RPS parameter may be positive while the other RPS parameter isnegative in some configurations. In general, the relative RPS parameterand the other RPS parameter may have different signs (e.g., negative orpositive).

The electronic device 702 may decode 1412 the current picture. This maybe performed, for example, on a decoder 712. The decoder 712 may be anHEVC type decoder 712.

FIG. 15 is a flow diagram illustrating a more specific configuration ofa method 1500 for decoding a picture on an electronic device. In oneconfiguration, the electronic device 702 may obtain 1502 a bitstream,obtain 1504 a current picture and obtain 1506 a relative RPS parameteras discussed previously in connection with FIG. 14.

The electronic device 702 may initialize 1508 an index value based onthe relative RPS parameter. For instance, Listing (14) shows one exampleof syntax for initializing 1508 an index value based on a relativenegative RPS parameter. Modifications to the syntax in accordance withthe systems and methods disclosed herein are denoted in bold. In someconfigurations, the approaches discussed in connection with Listing (14)may correspond to the approaches with Listing (4) discussed previously.

Listing (14) prev = 0; For (j=0; j < num_negative_pics; j++){   value =read( ) ;    RPS_deltaPoc(j) = prev − value − 1 ;    prev =RPS_deltaPoc(j);    ... } prev = (smallest absolute value of negativeRPS_deltaPoc) − 1; For (j=0; j < num_positive_pics; j++){    k =num_negative_pics+ j;   value = read( ) ;   RPS_deltaPoc(k) = value +1 + prev ;   prev = RPS_deltaPoc(k);   ... }

In some configurations, the index value may be “prev” in Listing (14),which may represent a previous index value. In this case, prev may beinitialized based on the smallest absolute value of negativeRPS_deltaPoc. RPS_deltaPOC refers to the RPS index values.

For example, suppose an RPS with reference indexes of [−2,−4, 2] isobtained by the electronic device 702. However, because of encoding asin the approach discussed in connection with FIG. 4, for example, thefirst positive RPS index received at the electronic device 702 may havea value of 0. The negative RPS index values are processed to generateone or more relative negative RPS parameters. Here, −2 is obtained asthe relative negative RPS parameter.

Continuing with this example, the index value is initialized 1508 basedon the relative negative RPS parameter. Listing (14) shows that prev maybe initialized based on the relative negative RPS parameter. Here, previs initialized based on the relative negative RPS parameter −2. In someinstances, the relative negative RPS parameter may be the smallestabsolute value taken from the set of negative RPS index values for acurrent picture. Thus, prev is initialized to the smallest absolutevalue taken from the set of negative RPS index values minus 1. In otherwords, prev=1 (e.g., |−2|−1).

The electronic device 702 may process 1510 another RPS parameter basedon the index value. For instance, returning to the above example, theelectronic device 702 may process 1510 another RPS parameter based onprev=1. In one instance, processing the positive RPS parameter mayinvolve employing RPS_deltaPoc(k)=value+1+prev. In this instance,RPS_deltaPoc(k) would equal a value (e.g., relative value of 0) plus 1plus prev (1). In other words, RPS_deltaPoc(k)=2. Thus, in this example,the positive RPS parameter generated is 2. Additionally, the generatedRPS index values for the current picture is [−2, −4, 2].

Once the RPS index values are obtained, the electronic device 702 maydecode 1512 the current picture. For example, the current picture mayuse the generated RPS index values for decoding.

In another example, suppose that the RPS has index values of [−4, −6,4]. Here, −4 would be obtained as the negative relative RPS parameter.The index value may be initialized to 3, or to the smallest absolutevalue of the negative RPS index values minus 1. In other words,prev=|−4|−1=3. Processing a positive RPS parameter value would result inan RPS index value of 4. Thus, in this example, the set of RPS indexvalues would be [−4, −6, 4].

FIG. 16 is a flow diagram illustrating another more specificconfiguration of a method 1600 for decoding a picture on an electronicdevice 702. The electronic device 702 may obtain 1602 a bitstream,obtain 1604 a current picture and obtain 1606 a relative RPS parameteras discussed previously in connection with FIG. 14.

The electronic device 702 may obtain 1608 a symmetric flag. In someconfigurations, the symmetric flag may be the same or similar to thesymmetric flag discussed previously. For example, the symmetric flag mayindicate that the relative negative RPS parameter corresponds to thepositive RPS parameter. For instance, given a set of RPS index values[−2, −6, 2, 4], the symmetric flag may indicate that the index value of−2 corresponds to the index value of 2.

Additionally, The symmetric flag may be sent in a bitstream 714 toindicate if the first positive RPS index value can be initialized with anegative RPS index value. For example, the symmetric flag may indicatethat the index value should be initialized to the smallest absolutevalue of negative RPS_deltaPoc values. Alternatively, the symmetry flagmay indicate not initializing the index value to the smallest absolutevalue of negative RPS_deltaPoc values. In this case, the index value maybe initialized to zero.

The electronic device 702 may initialize 1610 an index value based onthe relative negative RPS parameter and the symmetric flag. As oneexample, Listing (15) below illustrates one approach of initializing1610 an index value based on the relative negative RPS parameter and thesymmetric flag. Modifications to the syntax in accordance with thesystems and methods disclosed herein are denoted in bold.

Listing (15) prev = 0; For (j=0; j < num_negative_pics; j++){    value =read( ) ;     RPS_deltaPoc(j) = prev − value −1;     prev =RPS_deltaPoc(j);     ... } If (num_positive_pics > 0) {   symmetric_flag= read( );   if (symmetric_flag == true) {    RPS_deltaPoc(num_negative_pics) =  (smallest absolute value ofnegative RPS_deltaPoc) ; } prev = 0; For (j=0; j < num_positive_pics;j++){     k = num_negative_pics+ j;     if (j>0 || !symmetric_flag) {     value = read( ) ;      RPS_deltaPoc(k) = value + 1 + prev ;     }   prev = RPS_deltaPoc(k);     ... }

In one configuration of Listing (15), RPS_deltaPOC(num_negative_pics−1)stores the negative RPS index values. RPS_deltaPOC(num_negative_pics) toRPS_deltaPOC(num_negative_pics+num_positive_pics−1) stores the positiveRPS index values. In this case, RPS_deltaPoc(num_negative_pics) abovestores the first positive RPS index value. In other words,RPS_deltaPoc(num_negative_pics) refers to the first positive RPS indexvalue. First, the electronic device 702 verifies that there is at leastone positive RPS index value. Then the electronic device 702 determineswhether there is a symmetric flag and if the flag is set to true. If thethere is a symmetric flag set to true, the electronic device 702initializes 1610 an index value based on the relative negative RPSparameter. Thus, the initialization of the index value is based on boththe relative negative RPS parameter and the symmetric flag. In thisconfiguration, the index value of RPS_deltaPoc(num_negative_pics) isinitialized to the smallest absolute value of negative RPS_deltaPoc. Forexample, in the case of RPS index values [−8, −4, 4, 6], the firstpositive RPS index value may be initialized to 4, or the smallestabsolute value of negative RPS_deltaPoc.

The electronic device 702 may process 1612 one or more other RPSparameters based on the index value. For instance, with a positive indexvalue of 4, the electronic device 702 may be able to generate the secondpositive RPS index value of 6 in the example above. Further, theelectronic device 702 may skip processing at least one of the other RPSparameters when the index value is initialized. In other words, when thefirst positive RPS index value is initialized, processing the positiveRPS index values skips to the second positive RPS index value. Forinstance, in the example above, when the first positive RPS index valueis initialized to 4, processing skips to the second positive RPS indexvalue to generate a value of 6. Thus, in this example, the set of RPSindex values [−8, −4, 4, 6] is obtained.

In some cases, where there is only one positive RPS index value,processing of the positive RPS index valued may be skipped uponinitialization of the index value. For example, for index values [−4,−8, 4], the electronic device 702 may receive the RPS index values [−4,−8] and derive reference index values of [−4, −8, 4] for the currentpicture being decoded. Thus, by skipping processing of the firstpositive reference picture in each RPS for each current picture,processing time may be reduced.

Once the RPS index values are obtained, the electronic device 702 maydecode 1614 the current picture. For example, the current picture mayuse the processed RPS index values for decoding.

In another example, for the RPS index values [−2, −4, 2, 6], if thesymmetric_flag is true, for the positive RPS_deltaPOC values, the firstone must be equal to 2, as derived from the negative RPS index values.Other positive RPS index values may also be generated from theinitialized index value. For instance, processing of the positive RPSindex values may begin from the second positive RPS index value (e.g.,6) and may be based on the initialized index value of 2.

FIG. 17 is a flow diagram illustrating one configuration of a method1700 for signaling a reference picture set on an electronic device 302.In one known implementation, such as JCTVC-G637, pruning of referencepicture sets have been proposed. This implementation proposes anapproach to signal a number of short-term reference pictures that areomitted from a particular reference picture set (RPS) signaled in thepicture parameter set (PPS). For example, this approach is used whensome of the pictures signaled in RPS are not available because theybelong to a part of the first group of pictures (GOP) following a randomaccess point.

The systems and methods disclosed herein may provide one or moreadditional benefits for reference picture set (RPS) signaling andprediction. Additional benefits, such as improved efficiency from knownapproaches, may be obtained by inter RPS signaling and prediction whensignaling partial RPS. Further, the systems and methods disclosed hereinmay provide greater simplicity than known approaches.

In one configuration, an electronic device 302 may indicate 1702 apreviously signaled reference picture set (RPS) corresponding to acurrent picture. For example, the electronic device 302 may be anencoder 304. The encoder 304 may be an HEVC type encoder 304.

The current picture may belong to the first GOP after a random accesspoint. The previously signaled RPS may be sent at the random accesspoint as a part of the sequence parameter set (SPS) or as part ofanother parameter set (e.g. picture parameter set (PPS), adaptationparameter set (APS), video parameter set (VPS), etc.). Alternatively,the previously signaled RPS may be sent by using another element in thebitstream or may be obtained from some out-of-band mechanism. In anothercase, the current picture may not belong to the first GOP after a randomaccess point.

The previously signaled RPS may be identified by an index. For example,the index may be a RPS index, such as rps_idx[idx]. The index mayinclude corresponding values used in decoding the current picture.

The RPS corresponding to the current picture may be signaled by a deleteoperation on a previous signaled RPS. For example, a reference picturemay be lost and the decoder 712 may signal to the encoder 304, viafeedback, for the reference picture. However, the encoder 304 may nolonger use the lost picture as a reference frame. For example, this mayoccur when non-low delay conditions are present. By using a previouslysignaled RPS, the decoder 712 may be able to decode a current pictureusing one or more previously obtained reference pictures.

The electronic device 302 may set 1704 a first flag for one or morepictures in the previously signaled RPS. The first flag may be set 1704if the picture is to be used as a reference picture for the currentpicture. For example, the first flag may represent a Boolean value andmay be set to “1” if the picture is to be used as a reference picturefor the current picture. Alternatively, it may be appreciated that inanother configuration, the first flag may be set to “0” to indicate thatthe picture is to be used as a reference picture for the currentpicture.

As used herein, the term “positive,” when used in defining a flag,refers to the flag being set. For example, the first flag is positive ifit is set such that the picture is to be used as a reference picture forthe current picture. If the picture is not to be used as a referencepicture for the current picture, the flag may not be set (i.e., notpositive). It should be appreciated that, in some configurations, anegative value may alternatively be used when setting a flag. In someconfigurations, if the picture is not to be used as a reference picturefor the current picture, the flag may be omitted.

The previously signaled RPS may indicate multiple pictures to be used asreference pictures for the current picture. In this case, each pictureused as a reference picture for the current picture has its first flagset to be positive. In other words, for each reference picture in thepreviously signaled RPS, a corresponding flag is set if the referencepicture is to be used as a reference picture to decode the currentpicture. Also, in this case, each picture that is not used as areference picture for the current picture has its first flag set to benot positive.

The electronic device 302 may send 1704 a bitstream. The bitstream mayinclude the previously signaled RPS with a first flags set. Thebitstream may include multiple set second flags. Each set second flagmay correspond to a reference picture in the previously signaled RPSthat may be used as a reference picture for the current picture.

In some configurations, the bitstream may exclude a used by currentpicture flag value for the current RPS. In other words, the bitsream maynot include a used by current picture flag value for the current RPS.The used by current picture flag value for the current RPS may bepredicted and/or determined by an electronic device, such as a decoder712, based on corresponding values of one or more other referencepictures signaled in the previously signaled RPS.

In one approach, the previously signaled RPS may be signaled in thesequence parameter set (SPS). Listing (16) below provides one example ofsyntax that may be used in the sequence parameter set raw byte sequencepayload (RBSP) for signaling a reference picture set. Modifications tothe syntax in accordance with the systems and methods disclosed hereinare denoted in bold.

Listing (16) seq_parameter_set_rbsp( ) {  ... num_short_term_ref_pic_sets  for( i = 0; i <num_short_term_ref_pic_sets; i++)   short_term_ref_pic_set( i ) predicted_short_term_ref_pic_sets_present_flag if(predicted_short_term_ref_pic_sets_present_flag) {  num_predicted_short_term_ref_pic_sets_minus1   for(idx=0;idx<=num_short_term_ref_pic_sets_minus1; idx++){    rps_idx[idx]    for(j = 0; j < num_pics_in_rps_idx[idx]; j++ ) {     keep_pic_flag[j]    }  }  }   ... }

In Listing (16) num_short_term_ref_pic_sets specifies the number of fullshort-term reference picture sets that are specified in the pictureparameter set. The value of num_short_term_ref_pic_sets ranges from 0 to64, inclusive.

predicted_short_term_ref_pic_sets_present_flag equal to 1 specifies thata number of predicted short-term reference picture sets that are codedwith reference to full reference picture set exist in this sequenceparameter set. predicted_short_term_ref_pic_sets_present_flag equal to 0specifies that no predicted short-term reference picture sets that arecoded with reference to full reference picture set exist in thissequence parameter set.

In some configurations, a partial short-term reference picture set maybe part of a predicted short-term reference picture set. For example, apartial short-term reference picture sets present flag may be part of apredicted short-term reference picture sets present flag.

As shown in Listing (16), the previously signaled reference picture set(RPS) may be signaled by a flag, such aspredicted_short_term_ref_pic_sets_present_flag, in the SPS. A bit fieldin the SPS may correspond to each picture in the RPS to which theinter-RPS prediction applies. For example, the bit field may signalvalues of “1” or “0” to indicate whether to keep or discard acorresponding reference picture or corresponding reference picture valuefrom the previously signaled RPS.

num_predicted_short_term_ref_pic_sets_minus1 specifies the number ofpredicted short-term reference picture sets minus 1 that are specifiedin the sequence parameter set. The value ofnum_short_term_ref_pic_sets_minus1 ranges from 0 to 63, inclusive.

In one configuration, the decoder 712 may allocate space for a totalnumber of short-term reference picture sets since a coded video sequencemay include up to one short-term reference picture set explicitlysignaled in the slice headers of a current picture. For example, thetotal number of short-term reference picture sets may be determinedequalnum_short_term_ref_pic_sets+num_predicted_short_term_ref_pic_sets_minus1+1+1if predicted_short_term_ref_pic_sets_present_flag flag is equal to 1.The total number of short-term reference picture sets may be determinedequal num_short_term_ref_pic_sets+1 ifpredicted_short_term_ref_pic_sets_present_flag flag is equal to 0.

In this configuration, an explicitly signaled short-term referencepicture set in the slice header will always have an index equal to(num_short_term_ref_pic_sets+num_predicted_short_term_ref_pic_sets_minus1+1)in the list of short-term reference picture sets ifpredicted_short_term_ref_pic_sets_present_flag flag is equal to 1. Also,in this configuration, an explicitly signaled short-term referencepicture set in the slice header will always have an index equal to(num_short_term_ref_pic_sets+1) in the list of short-term referencepicture sets if predicted_short_term_ref_pic_sets_present_flag flag isequal to 0.

rps_idx[idx] specifies the index of that full short-term referencepicture set used to derive the predicted short-term reference pictureset signaled with index idx. num_pics_in_rps_idx[idx] specifies the sumof total number of negative pictures and total number of positivepictures included in the full short-term reference picture set in thepicture parameter set with index rps_idx[idx]. In some configurations,num_pics_in_rps_idx[idx] does not need to be signaled in the predictedshort-term reference picture set as it is already known fromcorresponding short-term reference picture set with index rps_idx[idx].

keep_pic_flag[j] equal to 1 specifies that the corresponding j^(th)short-term reference picture from the reference picture set in thesequence parameter set with index rps_idx[idx] should be kept to derivethis predicted reference picture set. keep_pic_flag[j] equal to 0specifies that the corresponding j^(th) short-term reference picturefrom the full short-term reference picture set in the picture parameterset with index rps_idx[idx] should be deleted (e.g., omitted) to derivethis predicted reference picture set.

In another approach, the previously signaled RPS may be signaled in thepicture parameter set (PPS). Listing (17) below provides one example ofsyntax that may be used in the picture parameter set raw byte sequencepayload (RBSP) signaling an inter reference picture. Modifications tothe syntax in accordance with the systems and methods disclosed hereinare denoted in bold.

Listing (17) pic_parameter_set_rbsp( ) {  pic_parameter_set_id seq_parameter_set_id  entropy_coding_mode_flag num_short_term_ref_pic_sets  for(idx = 0; idx <num_short_term_ref_pic_sets; idx++)   short_term_ref_pic_set( idx ) predicted_short_term_ref_pic_sets_present_flag if(predicted_short_term_ref_pic_sets_present_flag) {  num_predicted_short_term_ref_pic_sets_minus1   for(idx=0;idx<=num_short_term_ref_pic_sets_minus1; idx++){    rps_idx[idx]    for(j = 0; j < num_pics_in_rps_idx[idx]; j++ ) {     keep_pic_flag[j]    }  }  long_term_ref_pics_present_flag num_temporal_layer_switching_point_flags  for( i = 0; i <num_temporal_layer_switching_point_flags; i++ )  temporal_layer_switching_point_flag[ i ] num_ref_idx_l0_default_active_minus1 num_ref_idx_l1_default_active_minus1  pic_init_qp_minus26 /* relativeto 26 */  constrained_intra_pred_flag  slice_granularity shared_pps_info_enabled_flag  if( shared_pps_info_enabled_flag )   if(adaptive_loop_filter_enabled_flag )    alf_param( )  if(cu_qp_delta_enabled_flag )   max_cu_qp_delta_depth  rbsp_trailing_bits() }

FIG. 18 is a flow diagram illustrating another configuration of a method1800 for signaling a reference picture set on an electronic device 302.The electronic device 302 may signal one or more full reference picturesets (RPS). The RPSs may be sent in connection with a previously decodedpicture.

The electronic device 302 may determine 1904 if any predicted RPSs areto be signaled. If there are no predicted RPSs to be signaled, then theelectronic device 302 may signal 1806 no predicted RPSs. For example,the electronic device 302 may signal 1806predicted_short_term_ref_pic_sets_present_flag equal to “0.”

If there is one or more predicted RPSs to be signaled, then theelectronic device 302 may signal 1808 the one or more predicted RPSs.For example, the electronic device 302 may signal 1808predicted_short_term_ref_pic_sets_present_flag equal to “1.”

The electronic device 302 may send 1810 the number of predicted RPSs.For example, the electronic device 302 may send 1810num_predicted_short_term_ref_pic_sets_minus1.

For each predicted RPS that is signaled, the electronic device 302 maysend 1812 reference to a previously signaled full RPS. For example, theelectronic device 302 may send 1812 rps_idx[idx].

For each reference picture in the previously signaled RPS, theelectronic device 302 may signal 1814 if the reference picture is to beused as a reference picture for a current picture. For example, theelectronic device 302 may set a flag, such as a keep picture flag, thatindicates if the reference picture is to be used as a reference picturefor a current picture.

The electronic device 302 may exclude, or not send 1816, a used bycurrent picture flag for the current RPS. In other words, a used bycurrent picture flag value for the current RPS may not be indicated,signaled or encoded by the electronic device 302. The used by currentpicture flag value for the current RPS may be predicted and/ordetermined by an electronic device, such as a decoder 712.

In yet another approach, the previously signaled RPS may be signaled inthe short-term reference picture set. Listing (18) below provides oneexample of syntax that may be used in the short-term reference pictureset for signaling a reference picture set. Modifications to the syntaxin accordance with the systems and methods disclosed herein are denotedin bold.

Listing (18) short_term_ref_pic_set( idx ) { inter_ref_pic_set_prediction_flag  if(inter_ref_pic_set_prediction_flag ) {   rps_idx[idx]   for( j = 0; j <num_pics_in_rps_idx[idx]; j++ ) {    keep_pic_flag[j]   }  }  else {  num_negative_pics   num_positive_pics   for( i = 0; i <num_negative_pics; i++ ) {    delta_poc_s0_minus1[ i ]   used_by_curr_pic_s0_flag[ i ]   }   for( i = 0; i <num_positive_pics; i++ ) {    delta_poc_s1_minus1[ i ]   used_by_curr_pic_s1_flag[ i ]   }  } }

In still yet other approaches, the previously signaled RPS may besignaled in the adaptation parameter set (APS), video parameter set(VPS), slice header, etc. Further, the previously signaled RPS may besignaled by using another element in the bitstream.

FIG. 19 is a flow diagram illustrating a more specific configuration ofa method 1900 for signaling a reference picture set on an electronicdevice 302. The electronic device 302 may indicate 1902 that a predictedshort term RPS is signaled based on a previously signaled referencepicture set (RPS) using a predicted short-term reference picture setpresent flag. For example, the current picture using a predictedshort-term reference picture set present flag may be a Boolean valuethat indicates whether a previously signaled RPS is being signaled foruse in decoding a current picture. In some configurations, theelectronic device 302 may be an encoder 304. The encoder 304 may be anHEVC type encoder 304.

The current picture may belong to the first GOP after a random accesspoint and/or a delete operation on a previous signaled RPS. In eithercase, signaling a previously signaled RPS may provide increasedefficiency over currently known approaches.

The previously signaled RPS may correspond to an index. For example, thepreviously signaled RPS is signaled as an index into a set of previouslysignaled set of RPSs. Additionally, the index may be a RPS index, suchas rps_idx[idx]. The index may indicate the previously signaled RPS thatwill be used for predicting the RPS to be used for decoding the currentpicture.

The electronic device 302 may indicate 1904 a used by current pictureflag value of a second reference picture in the previously signaled RPS.The used by current picture flag value of a second reference picture maybe a value corresponding to the current picture. In other words, inindicating 1902 the previously signaled RPS, the electronic device 302may indicate one or more corresponding values of other referencepictures in the previously signaled RPS. The one or more correspondingvalues of other reference pictures in the previously signaled RPS may beused by current picture flag values for the other reference pictures inthe previously signaled RPS. The corresponding values may be indicated1904 by the electronic device 302 and encoded by an encoder 304.

The electronic device 302 may set 1906 a keep picture flag for a picturein the previously signaled RPS if the picture is to be used as areference picture for the current picture. In one configuration, thekeep picture flag may represent a bit field in the index correspondingto the previously signaled RPS. For example, each reference picture inthe RPS corresponding to the index rps_idx[idx] may be signaled a keeppicture flag that indicates whether to keep or discard the correspondingreference picture or corresponding reference picture value from thepreviously signaled RPS.

The electronic device 302 may exclude 1908 a used by current pictureflag for the current RPS. In other words, a used by current picture flagvalue for the current RPS may not be indicated, signaled or encoded bythe electronic device 302 (e.g., encoder 304). The used by currentpicture flag value for the current RPS may be predicted and/ordetermined by an electronic device, such as a decoder 712, based on theused by current picture flag values of one or more other pictures in thepreviously signaled RPS.

The electronic device 302 may send 1910 a bitstream. The bitstream mayinclude the previously signaled RPS with the predicted short-termreference picture set present flag set. The bitstream also may includemultiple set used by current picture flag corresponding to one or morereference pictures used to decode the current picture.

FIG. 20 is a flow diagram illustrating one configuration of a method2000 for predicting a reference picture set on an electronic device 302.In some known implementations, such as that given in HEVC Draft 6, anInter RPS prediction is supported. However, this approach is morecomplex and less efficient than the approaches described herein.

In another known implementation, such as JCTVC-I0347, syntax forinter-RPS prediction is provided. This syntax modifies HEVC draft 6syntax by merging delta_rps_sign and abs_delta_rps_minus1 into onesyntax element.

In yet another known implementation, such as JCT-G198, an interreference picture set prediction is proposed. The main use case in thisimplementation is reducing the number of bits to signal a new RPS byencoding it with respect to a previously signaled RPS. In thisimplementation, the signaling scheme sends a constant deltaRPS value.Additional fields can be sent to signal if a POC from previous RPSshould be skipped (e.g., should not be used).

In one configuration, an electronic device 302 obtains 2002 a bitsream.The electronic device 302 may obtain 2004 a current picture. Forexample, the current picture may be obtained from the bitstream.

The electronic device 302 may obtain a previously signaled referencepicture set (RPS). The previously signaled RPS may already be present onthe electronic device 302. In this case, the bitstream may have sent asignal for the current picture indicating the use of a previouslysignaled RPS.

In some configurations, the previously signaled RPS may be sent by anencoder 304 and received in the bitsream. For example, the bitstream mayinclude a first flag that indicates that a previously signaled RPS isbeing used to decode the current picture.

The previously signaled RPS may include one or more pictures used todecode previous pictures. The previously signaled RPS may correspond toan index. For example, the previously signaled RPS is signaled as anindex into a set of previously signaled set of RPSs. The index mayindicate corresponding reference pictures and/or values used in decodingthe current picture.

For each picture in the previously signaled RPS, the electronic devicemay obtain 2008 a first flag indication if the picture should be used asa reference picture for the current picture. The first flag may indicateif the picture is to be used as a reference picture for the currentpicture. For example, the first flag may represent a Boolean value andmay be set to “1” if the picture is to be used as a reference picturefor the current picture. Alternatively, it may be appreciated that inanother configuration, the first flag may be set to “0” to indicate thatthe picture is to be used as a reference picture for the currentpicture.

The first flag may be positive if it is set such that a picture in thepreviously signaled RPS is to be used as a reference picture for thecurrent picture. If the picture is not to be used as a reference picturefor the current picture, the flag may not be set (i.e., not positive).It should be appreciated that, in some configurations, a negative valuemay be used when setting the first flag. In the case of multiplepictures in the previously signaled RPS that are to be used as referencepictures for the current picture, each picture may have a correspondingfirst flag.

If the first flag is positive, the electronic device 302 may determine2010 a second flag from one corresponding value of the previouslysignaled RPS. In other words, if a picture in the previously signaledRPS is used as a reference picture for the current picture, theelectronic device 302 may infer a second flag value for the currentpictures. The second flag value may be inferred, predicted or determinedfrom second flag values of references pictures in the previouslysignaled RPS. The second flag may be a used by current picture flag.

The electronic device 302 may decode 2012 the current picture. This maybe performed, for example, on a decoder 712. The decoder 712 may be anHEVC type decoder 712.

FIG. 21 is a flow diagram illustrating another configuration of a method2100 for predicting a reference picture set on an electronic device 302.The electronic device 302 may obtain 2102 a signal for one or morepreviously received full reference picture sets (RPS). For example, thesignal may indicate a previously received RPSs used to decode a previouspicture.

The electronic device 302 may determine 2104 if there are any predictedRPSs signaled. If there are no signaled predicted RPSs, then theelectronic device 302 may obtain 2106 a signal indicating no predictedRPSs. For example, the electronic device 302 may obtain 2106 a signal,such as predicted_short_term_ref_pic_sets_present_flag equals “0.”

If there is one or more signaled predicted RPSs, then the electronicdevice 302 may obtain 2108 a signal indicating one or more predictedRPSs. For example, the electronic device 302 may obtain 2108 a signal,such as predicted_short_term_ref_pic_sets_present_flag equals “1.”

The electronic device 302 may obtain 2110 the number of predicted RPSs.For example, the electronic device 302 may obtain 2110num_predicted_short_term_ref_pic_sets_minus1.

For each predicted RPS that is signaled, the electronic device 302 mayobtain 2112 reference to a previously signaled full RPS. For example,the electronic device 302 may obtain 2112 rps_idx[idx].

For each reference picture in the previously signaled RPS, theelectronic device 302 may obtain 2114 a signal if the reference pictureis to be used as a reference picture for a current picture. For example,the electronic device 302 may obtain a flag, such as a keep pictureflag, that indicates if the reference picture is to be used as areference picture for a current picture.

The electronic device 302 may determine 2116 a used by current pictureflag value for a current picture from the value of a correspondingreference picture in the previously signaled RPS. The electronic device302 may determine 2116 the used by current picture flag value for thecurrent picture by inferring or predicting the used by current pictureflag value based on one or more corresponding (e.g., used by currentpicture flag value) values of the one or more reference picturessignaled in the previously RPS.

The electronic device 302 may decode 2118 the current picture based onthe used by current picture flag value. The used by current picture flagvalues may include the used by current picture flag value of the currentpicture.

FIG. 22 is a flow diagram illustrating a more specific configuration ofa method 2200 for predicting a reference picture set on an electronicdevice. The electronic device 302 may obtain 2202 a bitstream and obtain2204 a current picture. For example, the current picture may be obtainedfrom the bitstream.

The electronic device 302 may obtain 2204 a previously signaledreference picture set (RPS) based on a predicted short-term referencepictures present flag. In this manner, a RPS that has been previouslysignaled and is present on the electronic device 302, along withpreviously decoded reference pictures, may be used and/or reused todecode a current picture.

A predicted short-term reference pictures present flag may signal thepresence of previously signaled RPS. For example, if the predictedshort-term reference pictures present flag is set, then it may signal tothe electronic device 302 a previously signaled RPS. In someconfigurations, the presences of the predicted short-term referencepictures present flag may signal to the electronic device 302 apreviously signaled RPS.

The RPS may be sent separately from actual current or previous pictures.The RPS may be sent as part of the sequence parameter set (SPS), apicture parameter set (PPS), another parameter set and/or the sliceheader.

For each reference picture in the previously signaled RPS, theelectronic device 302 may obtain 2206 a keep picture flag indication ifthe picture should be used as a reference picture for the currentpicture. The keep picture flag indicates whether a picture in thepreviously signaled RPS is to be used as a reference picture in decodingthe current picture.

The electronic device 302 may obtain 2210 a corresponding value from thepreviously signaled RPS based on the keep picture flag. For example, ifthe keep picture flag indicates that a picture in the previouslysignaled RPS is to be used as a reference picture for the currentpicture, the electronic device 302 may obtain a corresponding value,such a used by current picture flag value, from the previously signaledRPS corresponding to the picture. In some configurations, the used bycurrent picture flag value for one or more reference pictures in thepreviously signaled RPS may be sent to the electronic device 302 by anencoder 304.

The electronic device 302 may determine 2212 a used by current pictureflag value for the current picture from the corresponding value of asecond reference picture in the previously signaled RPS. The electronicdevice 302 may determine the used by current picture flag value for thecurrent picture by inferring or predicting the used by current pictureflag value based on one or more corresponding (e.g., used by currentpicture flag value) values of the one or more reference pictures in thepreviously signaled RPS.

The electronic device 302 may decode 2214 the current picture based onthe used by current picture flag values. The used by current pictureflag values may include the used by current picture flag value of thecurrent picture as well as the used by current picture flag values ofcorresponding reference pictures in the signaled RPS.

-   -   The bitstream of the video may include a syntax structure that        is placed into logical data packets generally referred to as        Network Abstraction Layer (NAL) units. Each NAL unit includes a        NAL unit header, such as a two-byte NAL unit header (e.g., 16        bits), to identify the purpose of the associated data payload.        For example, each coded slice (and/or picture) may be coded in        one or more slice (and/or picture) NAL units. Other NAL units        may be included for other categories of data, such as for        example, supplemental enhancement information, coded slice of        temporal sub-layer access (TSA) picture, coded slice of        step-wise temporal sub-layer access (STSA) picture, coded slice        a non-TSA, non-STSA trailing picture, coded slice of broken link        access picture, coded slice of instantaneous decoded refresh        picture, coded slice of clean random access picture, coded slice        of decodable leading picture, coded slice of tagged for discard        picture, video parameter set, sequence parameter set, picture        parameter set, access unit delimiter, end of sequence, end of        bitstream, filler data, and/or sequence enhancement information        message. Table (5) illustrates one example of NAL unit codes and        NAL unit type classes. Other NAL unit types may be included, as        desired. It should also be understood that the NAL unit type        values for the NAL units shown in the Table (5) may be        reshuffled and reassigned. Also additional NAL unit types may be        added. Also some NAL unit types may be removed.

-   An intra random access point (IRAP) picture is a coded picture for    which each video coding layer NAL unit has nal_unit_type in the    range of BLA_W_LP to RSV_IRAP_VCL23, inclusive as shown in Table    (5). An IRAP picture contains only Intra coded (I) slices. An    instantaneous decoding refresh (IDR) picture is an IRAP picture for    which each video coding layer NAL unit has nal_unit_type equal to    IDR_W_RADL or IDR_N_LP as shown in Table (5). An instantaneous    decoding refresh (IDR) picture contains only I slices, and may be    the first picture in the bitstream in decoding order, or may appear    later in the bitstream. Each IDR picture is the first picture of a    coded video sequence (CVS) in decoding order. A broken link access    (BLA) picture is an IRAP picture for which each video coding layer    NAL unit has nal_unit_type equal to BLA_W_LP, BLA_W_RADL, or    BLA_N_LP as shown in Table (5). A BLA picture contains only I    slices, and may be the first picture in the bitstream in decoding    order, or may appear later in the bitstream. Each BLA picture begins    a new coded video sequence, and has the same effect on the decoding    process as an IDR picture. However, a BLA picture contains syntax    elements that specify a non-empty reference picture set.

-   When the current picture is an IRAP picture, the following applies:    -   If the current picture is an IDR picture, a BLA picture, the        first picture in the bitstream in decoding order, or the first        picture that follows an end of sequence NAL unit in decoding        order, a variable NoRasIOutputFlag is set equal to 1.    -   Otherwise, if some external means is available to set a variable        HandleCraAsBlaFlag to a value for the current picture, the        variable HandleCraAsBlaFlag is set equal to the value provided        by that external means and the variable NoRasIOutputFlag is set        equal to HandleCraAsBlaFlag.    -   Otherwise, the variable HandleCraAsBlaFlag is set equal to 0 and        the variable NoRasIOutputFlag is set equal to 0.        A random access decodable leading (RADL) picture is a coded        picture for which each VCL NAL unit has nal_unit_type equal to        RADL_R or RADL_N. All RADL pictures are leading pictures. RADL        pictures are not used as reference pictures for the decoding        process of trailing pictures of the same associated IRAP        picture. When present, all RADL pictures precede, in decoding        order, all trailing pictures of the same associated IRAP        picture. A random access skipped leading (RASL) picture is a        coded picture for which each VCL NAL unit has nal_unit_type        equal to RASL_R or RASL_N. All RASL pictures are leading        pictures of an associated BLA or CRA picture. When the        associated IRAP picture has NoRasIOutputFlag equal to 1, the        RASL picture is not output and may not be correctly decodable,        as the RASL picture may contain references to pictures that are        not present in the bitstream. RASL pictures are not used as        reference pictures for the decoding process of non-RASL        pictures. When present, all RASL pictures precede, in decoding        order, all trailing pictures of the same associated IRAP        picture.

TABLE (5) Name of Content of NAL unit and raw byte NAL unitnal_unit_type nal_unit_type sequence payload (RBSP) syntax structuretype class 0 TRAIL_N Coded slice segment of a non-TSA, non- Video 1TRAIL_R STSA trailing picture Coding slice_segment_layer_rbsp( ) Layer(VCL) 2 TSA_N Coded slice segment of a temporal sub-layer VCL 3 TSA_Raccess (TSA) picture slice_segment_layer_rbsp( ) 4 STSA_N Coded slicesegment of an Step-wise VCL 5 STSA_R Temporal sub-layer access (STSA)picture slice_segment_layer_rbsp( ) 6 RADL_N Coded slice segment of arandom access VCL 7 RADL_R decodable leading (RADL) pictureslice_segment_layer_rbsp( ) 8 RASL_N Coded slice segment of a randomaccess VCL 9 RASL_R skipped leading (RASL) pictureslice_segment_layer_rbsp( ) 10 RSV_VCL_N10 Reserved non-IRAP sub-layernon-reference VCL 12 RSV_VCL_N12 VCL NAL unit types 14 RSV_VCL_N14 11RSV_VCL_R11 Reserved non-IRAP sub-layer reference VCL VCL 13 RSV_VCL_R13NAL unit types 15 RSV_VCL_R15 16 BLA_W_LP Coded slice segment of abroken link access VCL 17 BLA_W_RADL (BLA) picture 18 BLA_N_LPslice_segment_layer_rbsp( ) 19 IDR_W_RADL Coded slice segment of aninstantaneous VCL 20 IDR_N_LP decoding refresh (IDR) pictureslice_segment_layer_rbsp( ) 21 CRA_NUT Coded slice segment of a cleanrandom VCL access (CRA) picture slice_segment_layer_rbsp( ) 22RSV_IRAP_VCL22 Reserved IRAP VCL NAL unit types VCL 23 RSV_IRAP_VCL23 24. . . 31 RSV_VCL24 . . . Reserved non-IRAP VCL NAL unit types VCLRSV_VCL31 32 VPS_NUT Video parameter set non-videovideo_parameter_set_rbsp( ) coding layer (non-VCL) 33 SPS_NUT Sequenceparameter set non-VCL seq_parameter_set_rbsp( ) 34 PPS_NUT Pictureparameter set non-VCL pic_parameter_set_rbsp( ) 35 AUD_NUT Access unitdelimiter non-VCL access_unit_delimiter_rbsp( ) 36 EOS_NUT End ofsequence non-VCL end_of_seq_rbsp( ) 37 EOB_NUT End of bitstream non-VCLend_of_bitstream_rbsp( ) 38 FD_NUT Filler data non-VCL filler_data_rbsp() 39 PREFIX_SEI_NUT Supplemental enhancement information non-VCL 40SUFFIX_SEI_NUT sei_rbsp( ) 41 . . . 47 RSV_NVCL41 . . . Reserved non-VCLRSV_NVCL47 48 . . . 63 UNSPEC48 . . . Unspecified non-VCL UNSPEC63

Referring to Table (6), the NAL unit header syntax may include two bytesof data, namely, 16 bits. The first bit is a “forbidden_zero_bit” whichis always set to zero at the start of a NAL unit. The next six bits is a“nal_unit_type” which specifies the type of raw byte sequence payloads(“RBSP”) data structure contained in the NAL unit as shown in Table (5).The next 6 bits is a “nuh_layer_id” which specify the identifier of thelayer. In some cases these six bits may be specified as“nuh_reserved_zero_(—)6 bits” instead. The nuh_reserved_zero_(—)6 bitsmay be equal to 0 in the base specification of the standard. In ascalable video coding and/or syntax extensions nuh_layer_id may specifythat this particular NAL unit belongs to the layer identified by thevalue of these 6 bits. The next syntax element is“nuh_temporal_id_plus1”. The nuh_temporal_id_plus1 minus 1 may specify atemporal identifier for the NAL unit. The variable temporal identifierTemporalId may be specified as TemporalId=nuh_temporal_id_plus1−1. Thetemporal identifier TemporalId is used to identify a temporal sub-layer.The variable HighestTid identifies the highest temporal sub-layer to bedecoded.

TABLE (6) Descriptor nal_unit_header( ) { forbidden_zero_bit f(1)nal_unit_type u(6) nuh_layer_id u(6) nuh_temporal_id_plus1 u(3) }

FIG. 23 illustrates a scenario which includes RADL and RASL picturetypes. In the FIG. 23 (b), for pictures with the same value ofPicOrderCntVal if a RASL picture exists at a lower layer which is adirect reference layer for the current layer and a RADL picture occursat the current layer then the RADL picture will not be decodable withthe current RPS restrictions which exist in HEVC draft standard. In thiscase currently using the The decoding process for inter-layer referencepicture set described in SHVC Draft Text 1 the RASL picture will beincluded in the inter-layer reference picture set—Ref PicSetInterLayerof RADL picture of enhancement layer. However this makes the RADLpicture at the enhancement layer un-decodable when random access isperformed at the associated CRA picture of the RADL picture. FIG. 23 (a)shows a scenario where such a problem does not arise. With respect toFIG. 23 (b) we describe a restriction on reference picture set whichresults in the RASL picture from base layer being not included in theRef PicSetInterLayer of the RADL picture from the enhancement layer.

In one embodiment the restriction is defined as follows: When thecurrent picture is a RADL picture, there shall be no picture included inRef PicSetStInterLayer that is any of the following:

-   -   A RASL picture    -   A picture that was generated by the decoding process for        generating unavailable reference pictures.        The decoding process for generating unavailable reference        picture is invoked once per coded picture when the current        picture is a BLA picture or is a CRA picture with        NoRasIOutputFlag equal to 1. This process is primarily specified        only for the specification of syntax constraints for RASL        pictures. The entire specification of the decoding process for        RASL pictures associated with an IRAP picture that has        NoRasIOutputFlag equal to 1 is only for purposes of specifying        constraints on the allowed syntax content of such RASL pictures.        During the decoding process, any RASL pictures associated with        an IRAP picture that has NoRasIOutputFlag equal to 1 may be        ignored, as these pictures are not specified for output and have        no effect on the decoding process of any other pictures that are        specified for output.

In another embodiment the restriction is defined as follows:

-   -   When the current picture is a RADL picture, there shall be no        RASL picture included in RefPicSetInterLayer.        In an additional laternative embodiment the restriction is        defined as follows:        When the current picture is a RADL picture, there shall be no        picture included in Ref PicSetStInterLayer that is any of the        following:    -   A RASL picture    -   A picture that was generated by the decoding process for        generating unavailable reference pictures.    -   A picture that precedes the associated IRAP picture in decoding        order.

FIG. 24 is a block diagram illustrating an inter-layer reference pictureset (RPS) creation module 2418. The inter-layer reference picture set(RPS) updating module 2418 may be part of a decoder 112 on an electronicdevice 102 b. The inter-layer reference picture set (RPS) updatingmodule 518 may be used by the decoder 112 to update an inter-layerreference picture set (RPS) (Ref PicSetInterLayer).

The inter-layer reference picture set (RPS) updating module 2418 mayinclude an inter-layer picture 2434. In one configuration, theinter-layer picture 2434 may be a non-IRAP (intra random access point)picture or a intra random access point (IRAP) picture. The inter-layerpicture 2434 may be a picture received from another electronic device102 a via a bitstream 110. The inter-layer reference picture set (RPS)updating module 2418 may determine whether the inter-layer picture 2434is added to the reference picture set (RPS).

The inter-layer picture 2434 may include a layer identifier(nuh_layer_id) 2436, a temporal identifier (TemporalId) 2438 and apicture order count (POC) 2453. An additional check is performed (todetermine if the inter-layer picture 2434 should be added to thereference picture set (RPS) 120) if the layer identifier (nuh_layer_id)2436 for the inter-layer picture 2434 corresponds to the layers whichare direct reference layers for the current layer and the picture ordercount (POC) 2453 of the inter-layer picture 2434 is equal to a pictureorder count (POC) 2461 of the current picture (nuh_layer_id) 2459.

The decoding process for inter-layer reference picture set is shownbelow in Table (7). In this pseudo-code if current picture is not a RADLpicture then a inter-layer reference picture from each of the layersthat are direct reference layers for the current picture and have thesame picture order count value as the current picture are added to theinter-layer reference picture set Ref PicSetInterLayer of the currentpicture. Also if the current picture is a RADL picture and theinter-layer reference picture is not a RASL picture and is also not anunavailable reference picture then inter-layer reference picture thathas the same picture order count value as the current picture is addedto the inter-layer reference picture set Ref PicSetInterLayer of thecurrent picture.

TABLE (7) for( i = 0, NumInterLayerRPSPics[LayerIdInVps[ nuh_layer_id ]]=0; i < NumDirectRefLayers[ LayerIdInVps[ nuh_layer_id ] ]; i++ ) { rPic= the picture with picture order count equal to PicOrderCnt and  nuh_layer_id equal to RefLayerId[ LayerIdInVps[ nuh_layer_id ][ i ] ] if(current picture is not RADL picture || (current picture is RADLpicture && (rPic != RASL picture && rPic != unavailable referencepicture)))   {RefPicSetInterLayer[ NumInterLayerRPSPics[LayerIdInVps[nuh_layer_id ] ] ] = rPic;   RefPicSetInterLayer[NumInterLayerRPSPics[LayerIdInVps[ nuh_layer_id ] ] ] is marked as “usedfor long-term reference”   NumInterLayerRPSPics[LayerIdInVps[nuh_layer_id ] ]++} }Where layer_id_in_nuh[i] specifies the value of the nuh_layer_id syntaxelement in VCL NAL units of the i-th layer. When not present, the valueof layer_id_in_nuh[i] is inferred to be equal to i. The variableLayerIdInVps[layer_id_in_nuh[i] ] is set equal to i.direct_dependency_flag[i][j] equal to 0 specifies that the layer withindex j is not a direct reference layer for the layer with index i.direct_dependency_flag[i][j] equal to 1 specifies that the layer withindex j may be a direct reference layer for the layer with index i. Whendirect_dependency_flag[i][j] is not present for i and j in the range of0 to vps_max_layers_minus1, it is inferred to be equal to 0.The variables NumDirectRefLayers[i] and RefLayerId[i][j] are derived asfollows:

for( i = 1; i <= vps_max_layers_minus1; i++ )  for( j = 0,NumDirectRefLayers[ i ] = 0; j < i; j++ )   if( direct_dependency_flag[i ][ j ] = = 1 )    RefLayerId[ i ][ NumDirectRefLayers[ i ]++ ] =   layer_id_in_nuh[ j ]Further embodiments for the decoding process for inter-layer referencepicture set is shown below in Table (8) and Table (9). They achieve thesame or similar result as Table (7) in decoding inter-layer referencepicture set but operate using a slightly different logic.

TABLE (8) for( i = 0, NumInterLayerRPSPics[LayerIdInVps[ nuh_layer_id ]]=0; i < NumDirectRefLayers[ LayerIdInVps[ nuh_layer_id ] ]; i++ ) { rPic= the picture with picture order count equal to PicOrderCnt and  nuh_layer_id equal to RefLayerId[ LayerIdInVps[ nuh_layer_id ][ i ] ] if(current picture is RADL picture && (rPic == RASL picture && rPic ==unavailable reference picture))   {//Do not add rPic toRefPicSetInterLayer}  else   {RefPicSetInterLayer[NumInterLayerRPSPics[LayerIdInVps[ nuh_layer_id ] ] ] = rPic;  RefPicSetInterLayer[ NumInterLayerRPSPics[LayerIdInVps[ nuh_layer_id ]] ] is marked as “used for long-term reference”  NumInterLayerRPSPics[LayerIdInVps[ nuh_layer_id ] ]++} }

TABLE (9) for( i = 0, NumInterLayerRPSPics[LayerIdInVps[ nuh_layer_id ]]=0; i < NumDirectRefLayers[ LayerIdInVps[ nuh_layer_id ] ]; i++ ) { rPic= the picture with picture order count equal to PicOrderCnt and  nuh_layer_id equal to RefLayerId[ LayerIdInVps[ nuh_layer_id ][ i ] ] if(current picture is RADL picture) {   if(rPic != RASL picture && rPic!= unavailable reference picture)  {RefPicSetInterLayer[NumInterLayerRPSPics[LayerIdInVps[ nuh_layer_id ] ] ] = rPic;RefPicSetInterLayer[ NumInterLayerRPSPics[LayerIdInVps[ nuh_layer_id ] ]] is marked as “used for long-term reference”  NumInterLayerRPSPics[LayerIdInVps[ nuh_layer_id ] ]++}   }  else  {RefPicSetInterLayer[ NumInterLayerRPSPics[LayerIdInVps[ nuh_layer_id] ] ] = rPic;   RefPicSetInterLayer[ NumInterLayerRPSPics[LayerIdInVps[nuh_layer_id ] ] ] is marked as “used for long-term reference”  NumInterLayerRPSPics[LayerIdInVps[ nuh_layer_id ] ]++} }

TABLE (10) NumPocTotalCurr = 0 for( i = 0; i < NumNegativePics[ StRpsIdx]; i++)  if(UsedByCurrPicS0[ StRpsIdx ][ i ] = = 1)   NumPocTotalCurr++for( i = 0; i < NumPositivePics[ StRpsIdx ]; i++) (xx) if(UsedByCurrPicS1[ StRpsIdx ][ i ] = = 1)   NumPocTotalCurr++ for( i =0; i < num_long_term_sps + num_long_term_pics; i++ )  if(UsedByCurrPicLt[ i ] = = 1)   NumPocTotalCurr++ NumPocTotalCurr +=NumInterLayerRPSPics[LayerIdInVps[ nuh_layer_id]]where StRpsIdx is the index of short-term RPSnum_long_term_sps specifies the number of entries in the long-term RPSof the current picture that are derived based the candidate long-termreference pictures specified in the active SPS. The value ofnum_long_term_sps shall be in the range of 0 tonum_long_term_ref_pics_sps, inclusive. When not present, the value ofnum_long_term_sps is inferred to be equal to 0.num_long_term_pics specifies the number of entries in the long-term RPSof the current picture that are directly signalled in the slice header.When not present, the value of num_long_term_pics is inferred to beequal to 0.lt_idx_sps[i] specifies an index, into the list of candidate long-termreference pictures specified in the active SPS, of the i-th entry in thelong-term RPS of the current picture. The number of bits used torepresent lt_idx_sps[i] is equal toCeil(Log2(num_long_term_ref_pics_sps)). When not present, the value oflt_idx_sps[i] is inferred to be equal to 0. The value of lt_idx_sps[i]shall be in the range of 0 to num_long_term_ref_pics_sps−1, inclusive.poc_lsb_lt[i] specifies the value of the picture order count moduloMaxPicOrderCntLsb of the i-th entry in the long-term RPS of the currentpicture.used_by_curr_pic_lt_flag[i] equal to 0 specifies that the i-th entry inthe long-term RPS of the current picture is not used for reference bythe current picture.The variables PocLsbLt[i] and UsedByCurrPicLt[i] are derived as follows:

-   -   If i is less than num_long_term_sps, PocLsbLt[i] is set equal to        lt_ref_pic_poc_lsb_sps[lt_idx_sps[i]] and UsedByCurrPicLt[i] is        set equal to used_by_curr_pic_lt_sps_flag[lt_idx_sps[i] ].    -   Otherwise, PocLsbLt[i] is set equal to poc_lsb_lt[i] and        UsedByCurrPicLt[i] is set equal to used_by_curr_pic_lt_flag[i].        num_negative_pics specifies the number of entries in the        stRpsIdx-th candidate short-term RPS that have picture order        count values less than the picture order count value of the        current picture. The value of num_negative_pics shall be in the        range of 0 to        sps_max_dec_pic_buffering_minus1[sps_max_sub_layers_minus1],        inclusive.

num_positive_pics specifies the number of entries in the stRpsIdx-thcandidate short-term RPS that have picture order count values greaterthan the picture order count value of the current picture. The value ofnum_positive_pics shall be in the range of 0 tosps_max_dec_pic_buffering_minus1[sps_max_sub_layers_minus1]−num_negative_pics,inclusive.

delta_poc_s0_minus1[i] plus 1, when i is equal to 0, specifies thedifference between the picture order count values of the current pictureand i-th entry in the stRpsIdx-th candidate short-term RPS that haspicture order count value less than that of the current picture, or,when i is greater than 0, specifies the difference between the pictureorder count values of the i-th entry and the (i+1)-th entry in thestRpsIdx-th candidate short-term RPS that have picture order countvalues less than the picture order count value of the current picture.The value of delta_poc_s0_minus1[i] shall be in the range of 0 to 2¹⁵−1,inclusive.used_by_curr_pic_s0_flag[i] equal to 0 specifies that the i-th entry inthe stRpsIdx-th candidate short-term RPS that has picture order countvalue less than that of the current picture is not used for reference bythe current picture.delta_poc_s1_minus1[i] plus 1, when i is equal to 0, specifies thedifference between the picture order count values of the current pictureand the i-th entry in the stRpsIdx-th candidate short-term RPS that haspicture order count value greater than that of the current picture, or,when i is greater than 0, specifies the difference between the pictureorder count values of the (i+1)-th entry and i-th entry in the currentcandidate short-term RPS that have picture order count values greaterthan the picture order count value of the current picture. The value ofdelta_poc_s1_minus1[i] shall be in the range of 0 to 2¹⁵−1, inclusive.used_by_curr_pic_s1_flag[i] equal to 0 specifies that the i-th entry inthe current candidate short-term RPS that has picture order count valuegreater than that of the current picture is not used for reference bythe current picture.When inter_ref_pic_set_prediction_flag is equal to 0, the variablesNumNegativePics[stRpsIdx], NumPositivePics[stRpsIdx],UsedByCurrPicS0[stRpsIdx][i], UsedByCurrPicS1[stRpsIdx][i],DeltaPocS0[stRpsIdx][i], and DeltaPocS1[stRpsIdx][i] are derived asfollows:

NumNegativePics[stRpsIdx]=num_negative_pics

NumPositivePics[stRpsIdx]=num_positive_pics

UsedByCurrPicS0[stRpsIdx][i]=used_by_curr_pic_(—) s0_flag[i]

UsedByCurrPicS1[stRpsIdx][i]=used_by_curr_pic_(—) s1_flag[i]

-   -   If i is equal to 0, the following applies:

DeltaPocS0[stRpsIdx][i]=−(delta_poc_(—) s0_minus1[i]+1)

DeltaPocS1[stRpsIdx][i]=delta_poc_(—) s1_minus1[i]+1

-   -   Otherwise, the following applies:

DeltaPocS0[stRpsIdx][i]=DeltaPocS0[stRpsIdx][i−1]−(delta_poc_(—)s0_minus1[i]+1)

DeltaPocS1[stRpsIdx][i]=DeltaPocS1[stRpsIdx][i−1]+(delta_poc_(—)s1_minus1[i]+1)

The variable NumDeltaPocs[stRpsIdx] is derived as follows:

NumDeltaPocs[stRpsIdx]=NumNegativePics[stRpsIdx]+NumPositivePics[stRpsIdx]

A change in the restriction on the NumPocTotalCurr value of CRA picturesis described below.Because Inter-layer prediction is allowed for CRA NAL units withnuh_layer_id greater than zero, while inter prediction is disallowed.Since NumPocTotalCurr is calculated as described in Table (10) toinclude the number of pictures in inter-layer reference picture set, wepropose following change in the restriction on the NumPocTotalCurr valueof CRA pictures. The variable NumPocTotalCurr is derived as specified inTable (10). It is a requirement of bitstream conformance that thefollowing applies to the value of NumPocTotalCurr:

-   -   If the current picture is a BLA picture or a CRA picture with        nuh_layer_id equal to 0, the value of NumPocTotalCurr shall be        equal to 0.    -   Otherwise, when the current picture contains a P or B slice, or        is a CRA picture with nuh_layer_id greater than 0 the value of        NumPocTotalCurr shall not be equal to 0.        In a variant embodiment it is a requirement of bitstream        conformance that the following applies to the value of        NumPocTotalCurr:    -   If the current picture is a BLA picture or a CRA picture with        nuh_layer_id equal to 0, the value of NumPocTotalCurr shall be        equal to 0.    -   Otherwise, when the current picture contains a P or B slice, the        value of NumPocTotalCurr shall not be equal to 0.        In yet other embodiments if the current picture is a CRA picture        with nuh_layer_id value greater than zero then NumPocTotalCurr        may or may not be equal to zero.

The term “computer-readable medium” refers to any available medium thatcan be accessed by a computer or a processor. The term“computer-readable medium,” as used herein, may denote a computer-and/or processor-readable medium that is non-transitory and tangible. Byway of example, and not limitation, a computer-readable orprocessor-readable medium may comprise RAM, ROM, EEPROM, CD-ROM or otheroptical disk storage, magnetic disk storage or other magnetic storagedevices, or any other medium that can be used to carry or store desiredprogram code in the form of instructions or data structures and that canbe accessed by a computer or processor. Disk and disc, as used herein,includes compact disc (CD), laser disc, optical disc, digital versatiledisc (DVD), floppy disk and Blu-ray® disc where disks usually reproducedata magnetically, while discs reproduce data optically with lasers.

It should be noted that one or more of the methods described herein maybe implemented in and/or performed using hardware. For example, one ormore of the methods or approaches described herein may be implemented inand/or realized using a chipset, an ASIC, a large-scale integratedcircuit (LSI) or integrated circuit, etc.

Each of the methods disclosed herein comprises one or more steps oractions for achieving the described method. The method steps and/oractions may be interchanged with one another and/or combined into asingle step without departing from the scope of the claims. In otherwords, unless a specific order of steps or actions is required forproper operation of the method that is being described, the order and/oruse of specific steps and/or actions may be modified without departingfrom the scope of the claims.

It is to be understood that the claims are not limited to the preciseconfiguration and components illustrated above. Various modifications,changes and variations may be made in the arrangement, operation anddetails of the systems, methods, and apparatus described herein withoutdeparting from the scope of the claims.

1. (canceled)
 2. A method for decoding a reference picture set on anelectronic device, comprising: obtaining a bitstream; obtaining an interlayer reference picture set (RPS) from the bitstream, wherein the interlayer reference picture set does not include a random access skippedleading (RASL) picture if the current picture is a random accessdecodable leading (RADL) picture.